gmane.org.w3c.whatwg.discuss

Re: Adding features needed for WebGL to ImageBitmap

Rik Cabanier — 2013-06-20T03:19:05


yeah. I'm still fuzzy about what exactly you're trying to do. Is this
specified anywhere?



I think that sounds good. That way implementations aren't forced to
premultiply the alpha.
Maybe not passing premultipiedAlpha in the dictionary (= leave it
undefined) will signal this.



Yes, the canvas spec calls this out:

Due to the lossy nature of converting to and from premultiplied alpha color
values, pixels that have just been set using
putImageDataHD()<http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html#dom-context-2d-putimagedatahd>
might
be returned to an equivalent
getImageDataHD()<http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html#dom-context-2d-getimagedatahd>
as
different values.

I think this is really unfortunate...

Re: Proposal: createImageBitmap should return a "Promise" instead of using a callback

Anne van Kesteren — 2013-06-20T02:18:07

I think something like

interface ImageBitmap {
  static Promise create(ImageBitmapSource image, optional long sx,
long sy, long sw, long sh);
};

would be much nicer.


--
http://annevankesteren.nl/

Re: Adding features needed for WebGL to ImageBitmap

Gregg Tavares — 2013-06-19T23:47:43


Yes, that's what I meant. I think I choose bad labels. the intent of the
colorspaceConversion flag is

colorspaceConversion: true   = browser does whatever it thinks is best for
color images.
colorspaceConversion: false  = give me the bits in the image file. Don't
manipulate them with either embedded color data or local machine gamma
corrections or anything else.

So maybe a better name?

for premultipiedAlpha again, maybe there are 3 options needed

1) do whatever is best for drawing with drawImage for perf
2) give me the data with premutlipied alpha
3) give me the data with non-premultied alpha.

It's possible that #1 is not needed as maybe GPU code can use different
blend modes for drawImage with non-premultpiled alpha. It's just my
understanding that at least in Chrome all images are loaded premultiplied.
In fact I don't think you can get non-premultipied data from canvas. At
least this does not make it appear that way

    c = document.createElement("canvas");
    ctx = c.getContext("2d");
    i = ctx

Re: Challenging canvas.supportsContext

Benoit Jacob — 2013-06-19T23:22:20

2013/6/19 Kenneth Russell <kbr-hpIqsD4AKlfQT0dZR+AlfA< at >public.gmane.org>


In Mozilla code, we fail WebGL context creation if any GL error occurs
during initialization, where we have to make a number of specific OpenGL
calls (e.g. querying many constants, enabling point sprites...). So it is
perfectly possible for WebGL specifically to fail on a given device where
OpenGL compositing works, without being specifically blacklisted, and
currently we can rely on these automatic checks rather than having to
curate a blacklist for these problems, which is very nice.

Another way in which WebGL specifically can fail, is that some driver bugs
cause errors that we may want to ignore in our compositor code but not in
WebGL. To give an example, on many Vivante GPU drivers, which are very
common in Chinese mobile devices, the first eglMakeCurrent call on a newly
created context can return false without any actual EGL error [1]. A
browser may want to ignore this error in its compositor to be able to run
nonetheless on such

Re: Adding features needed for WebGL to ImageBitmap

Rik Cabanier — 2013-06-19T23:03:33

OK, I see what you're trying to accomplish. You want to pass
non-premultiplied data and color converted (from sRGB to monitor) pixels to
WebGL
I think your API looks fine, except that the defaults should all be false...

Re: Adding features needed for WebGL to ImageBitmap

Rik Cabanier — 2013-06-19T23:01:28


You might have implemented it this way under the hood, but the Canvas spec
doesn't say anything about this.
It just says that data needs to be in non-premultiplied alpha when you read
or write it.



I'm unsure if I follow. Some browsers will convert images to sRGB if they
have embedded profiles. However, if the images is sRGB already, what other
color conversions do you need? All compositing should happen in sRGB...

Re: Challenging canvas.supportsContext

Kenneth Russell — 2013-06-19T22:56:24

In my experience, in Chromium, creation of the underlying OpenGL
context for a WebGLRenderingContext almost never fails in isolation.
Instead, more general failures happen such as the GPU process failing
to boot, or creation of all OpenGL contexts (including the
compositor's) failing. These failures would be detected before the app
calls supportsContext('webgl'). For this reason I believe
supportsContext's answer can be highly accurate in almost every
situation.


This is a specious question. My point is that the answer can be
accurate enough to be useful, and I'm personally willing to sign up to
make the implementation follow a stricter spec.

-Ken

Re: Challenging canvas.supportsContext

James Robinson — 2013-06-19T22:39:06


I'm assuming you have (1) and (3) flipped here and both supportsContext()
and getContext() support additional attributes to dictate whether a
software-provided context can be supplied.  In that case, in order to write
correct code I'd still have to attempt to fetch the contexts before using
them, i.e.:

var ctx = canvas.getContext('webgl', { 'allowSoftware': false});
if (ctx) {
  doPreference1(ctx);
  return;
}
ctx = canvas.getContext('2d', {'allowSoftware': false});
if (ctx) {
  doPreference2(ctx);
// etc

how could I simplify this code using supportsContext() ?





It seems overwhelmingly likely that one of the state changes that might
affect the result will be attempting to instantiate a real context.



I don't see how supportsContext() could be as accurate as getContext()
without doing all of the work getContext() does.  If it's not 100%
accurate, when is it useful?

- James

Re: Challenging canvas.supportsContext

Tab Atkins Jr. — 2013-06-19T22:36:03

It seems like this falls into the canPlayType() bucket, where you can
definitely give negatives, but your positives are at best hopeful.
Maybe we should just have it do the same thing, and return "maybe" or
"probably" rather than a boolean true?

~TJ

Re: Adding features needed for WebGL to ImageBitmap

Gregg Tavares — 2013-06-19T22:24:54


AFAIK the canvas API expects all images to be premultiplied. Certainly in
WebKit and Blink images used in the canvas and displayed in the img tag are
loaded premultiplied which is why we had to add the option on WebGL since
we needed those images before they had lost data.



No, games often animate texture coordinates and other things making it far
more complicated. There are ways to work around this issue in code yes,
they often require a ton of work.

Most professional game engines pre-process the textures and flip them
offline but that doesn't help when you're downloading models off say
http://sketchup.google.com/3dwarehouse/



Maybe but that's not relevant to ImageBitmap is it? The point here is we
want the ImageBitmap to give us the data in the format we need. It's
designed to be async so it can do this but it we can't prevent it from
applying colorspace conversions.
Some browsers did that for regular img tags which pointed out the original
problem. The browser can't guess how the image is going to

Re: Challenging canvas.supportsContext

Kenneth Russell — 2013-06-19T22:24:25

Any application which has a complex set of fallback paths. For example,

  - Preference 1: supportsContext('webgl', { softwareRendered: true })
  - Preference 2: supportsContext('2d', { gpuAccelerated: true })
  - Preference 3: supportsContext('webgl', { softwareRendered: false })
  - Fallback: 2D canvas

I agree that ideally, if supportsContext returns true then -- without
any other state changes that might affect supportsContext's result --
getContext should return a valid rendering context. It's simply
impossible to guarantee this correspondence 100% of the time, but if
supportsContext's spec were tightened somehow, and conformance tests
were added which asserted consistent results between supportsContext
and getContext, would that address your concern?

-Ken

Re: Adding features needed for WebGL to ImageBitmap

Rik Cabanier — 2013-06-19T22:13:23


When would premultipliedAlpha ever be true?
2D Canvas always works with non-premultiplied alpha in its APIs.



Couldn't you just draw upside down?



Shouldn't the color space conversion happen when the final canvas bit are
blitted to the screen?
It seems like you should never do it during compositing since you could get
double conversions.




 Like Tab said, it's fine to implement it that way.
Be aware that you might have to do some work in your idl compiler since I
*think* there are no other APIs (in Blink) that take a dictionary.

Re: Challenging canvas.supportsContext

James Robinson — 2013-06-19T22:06:41


That's true, but the answer still doesn't promise anything about what
getContext() will do.  It may still return null and code will have to check
for that.  What's the use case for calling supportsContext() without
calling getContext()?

- James

Re: Challenging canvas.supportsContext

Kenneth Russell — 2013-06-19T22:04:32

supportsContext() can give a much more accurate answer than
!!window.WebGLRenderingContext. I can only speak for Chromium, but in
that browser, it can take into account factors such as whether the GPU
sub-process was able to start, whether WebGL is blacklisted on the
current card, whether WebGL is disabled on the current domain due to
previous GPU resets, and whether WebGL initialization succeeded on any
other page. All of these checks can be done without the heavyweight
operation of actually creating an OpenGL context.

-Ken

Re: Adding features needed for WebGL to ImageBitmap

Tab Atkins Jr. — 2013-06-19T22:01:42

Passing an options object is the standard way.  Don't look to XHR for
*anything* about precedents.

~TJ

Adding features needed for WebGL to ImageBitmap

Gregg Tavares — 2013-06-19T21:47:18

In order for ImageBitmap to be useful for WebGL we need more options

Specifically

premultipliedAlpha: true/false (default true)
Nearly all GL games use non-premultipiled alpha textures. So all those
games people want to port to WebGL will require non-premultipied textures.
Often in games the alpha might not even be used for alpha but rather for
glow maps or specular maps or the other kinds of data.

flipY: true/false (default false)
Nearly all 3D modeling apps expect the bottom left pixel to be the first
pixel in a texture so many 3D engines flip the textures on load. WebGL
provides this option but it takes time and memory to flip a large image
therefore it would be nice if that flip happened before the callback from
ImageBitmap

colorspaceConversion: true/false (default true)
Some browsers apply color space conversion to match monitor settings.
That's fine for images with color but WebGL apps often load heightmaps,
normalmaps, lightmaps, global illumination maps and many other kinds of
data through images

Re: Challenging canvas.supportsContext

Paul Irish — 2013-06-19T21:27:41

On the Modernizr side, an old version did indeed create a context for
feature detection. For the past two years we have advocated the soft
`"WebGLRenderingContext" in window` test instead. There, of course, is some
gap between the results of that detect and how successful a getContext call
will be, but I don't have data those false positive rates.

A use case of a Modernizr user is to detect WebGL and show a browser prompt
if it's not detected.  From my understanding nearly no developer re-use the
same context for both detection and use.

I asked mrdoob, developer of three.js if supportsContext would be useful to
the project.. He said they currently create 2 webgl contexts "because,
API-wise is not easy to reuse the context that you created for checking if
it was available. Well, it's not pretty, at least"

I know the Chrome Web Store had a usecase for detection without context
creation: they wanted to not show apps in the marketplace that couldn't run
on the user's machine.

I agree that supportsContext is

Re: Challenging canvas.supportsContext

Tab Atkins Jr. — 2013-06-19T21:26:57

Hmm, I guess I agree.  That sucks, but you're right - it's much more
important that you *actually* be able to generate a context when the
feature-detect says you can.

~TJ

Re: Challenging canvas.supportsContext

Brandon Benvie — 2013-06-19T21:20:40

I don't have an example, I was just explaining how Mozernizr is often used.


Given this, it would seem supportsContext is completely useless. The 
whole purpose of a feature detection check is to detect if a feature 
actually works or not. Accuracy is more important than cost.

Re: Challenging canvas.supportsContext

Benoit Jacob — 2013-06-19T21:12:54

2013/6/19 Boris Zbarsky <bzbarsky-3s7WtUTddSA< at >public.gmane.org>


Yes, it seems that supportsContext being under-specified allows for
confusion to happen, as it is given different meanings in different emails
in this thread, that are mutually incompatible:

From Tab's 1st email:


From Tab's 2nd email:


The incompatibility is that the second quote's requirement can only be met
if supportsContext("webgl") actually creates an OpenGL context --- which is
incompatible with the first quote, which requires supportsContext to be
significantly quicker than getContext, which can only be achieved by not
actually creating an OpenGL context.

(Replace "OpenGL context" by "Direct3D device" or whichever concept applies
to the operating system at hand).

Benoit

Re: Challenging canvas.supportsContext

James Robinson — 2013-06-19T21:05:12


What would a page using Modernizr (or other library) to feature detect
WebGL do if the supportsContext('webgl') call succeeds but the later
getContext('webgl') call fails?

I'm also failing to see the utility of the supportsContext() call.  It's
impossible for a browser to promise that supportsContext('webgl') implies
that getContext('webgl') will succeed without doing all of the expensive
work, so any correctly authored page will have to handle a
getContext('webgl') failure anyway.

- James