I’ve come to greatly respect OpenGL and for which it stands as I discussed in my prior post. OpenGL was recently added to our awareness for a project for work at a time I was dabbling with it at home. These parallel lines of development brought excitement and mutual benefits for each locale. The project I am working on at work depends on what is inherently gaming technology even though you would think so to look at it.
I needed the ability to render the scene to a secondary framebuffer object (or FBO) due to the way the application incrementally draws a picture over time. Successive scenes merely draw the remaining portions whilst being merged with previous fragments rendered in the framebuffer. Think of the way Google Earth draws the image.
Anyway, the code worked fine on OpenGL 3.3 but for some reason the FBO failed the completeness test on OpenGL ES 2 under iOS. After much banging of head-to-wall I found the reason was due to the fact I was creating the secondary FBO ultimately during GLKViewController’s viewDidLoad call. This puzzled me as I already have a OpenGL context at this time due to a:
// We want OpenGL ES 2 thank-you very much
self.context = [[EAGLContext alloc] initWithAPI:kEAGLRenderingAPIOpenGLES2];
It seems creating a secondary framebuffer will fail if the size differs to the main framebuffer size. The best way to create secondary framebuffers is to wait for a viewWillLayoutSubviews notification. By this time your GLKit view is already resized so all you need to do is query the new size and create an appropriate FBO.
...MyDualFramebufferView _myView; | |
- (void)viewWillLayoutSubviews | |
{ | |
CGRect rc = self.view.bounds; | |
GLfloat scale=1; | |
if ([[UIScreen mainScreen] respondsToSelector:@selector(scale)]) | |
{ | |
scale = [[UIScreen mainScreen] scale] ; | |
} | |
_myView->Resize (rc.size.width, rc.size.height, scale); // myView is anything you like | |
[((GLKView *) self.view) bindDrawable]; // reset to main framebuffer because we changed it when we made a 2nd framebuffer | |
} |
The above controller calls into the following view for creating the framebuffer
GLsizei cxFBO; | |
GLsizei cyFBO; | |
GLuint _frameBufferID; | |
GLuint _colorRenderBuffer; | |
GLuint _depthRenderBuffer; | |
GLuint _fboTextureID; | |
void MyDualFramebufferView::Resize(GLfloat width, GLfloat height, GLfloat scale) | |
{ | |
_scale = scale; | |
_aspect = (GLfloat) width / height; | |
_screen = Vec2 (width, height); // Vec2 is actually a glm::vec2 | |
cxFBO=_screen.x * scale; | |
cyFBO=_screen.y * scale; | |
CreateRenderToTextureFBO(); | |
} | |
void MyDualFramebufferView::CreateRenderToTextureFBO() | |
{ | |
if (_frameBufferID != 0) | |
{ | |
glDeleteRenderbuffers(1, &_depthRenderBuffer); | |
glDeleteRenderbuffers(1, &_colorRenderBuffer); | |
glDeleteFramebuffers(1, &_frameBufferID); | |
cout << "Recreating FBO" << endl; | |
} | |
GLint maxTextureSize; | |
glGetIntegerv(GL_MAX_TEXTURE_SIZE, &maxTextureSize); | |
if ((cxFBO > maxTextureSize) || | |
(cyFBO > maxTextureSize)) | |
{ | |
throw "Requested size of frame buffer exceeds maximum"; | |
} | |
glGenFramebuffers(1, &_frameBufferID); | |
glGenRenderbuffers(1, &_depthRenderBuffer); | |
glGenTextures(1, &_fboTextureID); | |
// texture | |
glBindTexture(GL_TEXTURE_2D, _fboTextureID); | |
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); | |
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); | |
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR); | |
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST); | |
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, cxFBO, cyFBO, 0, GL_RGB, GL_UNSIGNED_BYTE, NULL); | |
// depth buffer | |
glBindRenderbuffer(GL_RENDERBUFFER, _depthRenderBuffer); | |
glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH_COMPONENT16, cxFBO, cyFBO); | |
// frame buffer | |
glBindFramebuffer(GL_FRAMEBUFFER, _frameBufferID); | |
// attachments | |
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, _fboTextureID, 0); | |
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, _depthRenderBuffer); | |
glBindFramebuffer(GL_FRAMEBUFFER, _frameBufferID); | |
GLenum status=glCheckFramebufferStatus(GL_FRAMEBUFFER); | |
if (status != GL_FRAMEBUFFER_COMPLETE) | |
{ | |
throw "Framebuffer is not complete"; | |
} | |
glBindTexture(GL_TEXTURE_2D, 0); | |
glBindFramebuffer(GL_FRAMEBUFFER, 0); | |
} | |
To draw, you must:
- Have the controller call into your view
- Bind to your secondary FBO
- Draw
- Return to the controller
- Have the controller rebind to the main FBO
- Call into your view again to draw the offscreen texture
- (void)glkView:(GLKView *)view drawInRect:(CGRect)rect | |
{ | |
_myView->DrawToBackbuffer(); // this will draw in 2nd FBO | |
[((GLKView *) self.view) bindDrawable]; // reset to main framebuffer | |
_myView->RenderToMainscreen(); | |
} |
void MyDualFramebufferView::DrawToBackbuffer(IDrawable& drawable) | |
{ | |
// use off screen FBO | |
glBindFramebuffer(GL_FRAMEBUFFER, _frameBufferID); | |
glViewport(0,0, cxFBO, cyFBO); | |
glClearColor(0, 0, 0, 1.0f); | |
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); | |
. | |
. | |
. | |
} |
- The final step is to draw the screen quad (steps 5-6 above)
void MyDualFramebufferView::RenderToMainscreen(IDrawable& drawable) | |
{ | |
// switch back to main | |
glClearColor(1, 0, 0, 1.0f); | |
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); | |
// trigger mipmaps generation explicitly | |
glBindTexture(GL_TEXTURE_2D, _fboTextureID); | |
glGenerateMipmap(GL_TEXTURE_2D); | |
glBindTexture(GL_TEXTURE_2D, 0); | |
glActiveTexture ( GL_TEXTURE0 ); | |
glUniform1i ( _basicArgs.Uniforms[UNIFORM_BASIC_SAMPLER0], 0 ); | |
glVertexAttribPointer (ATTRIBUTE_BASIC_POSITION, 3, GL_FLOAT, GL_FALSE, 5 * sizeof(GLfloat), _screenQuadVertices ); | |
glVertexAttribPointer (ATTRIBUTE_BASIC_TEX_COORD, 2, GL_FLOAT, GL_FALSE, 5 * sizeof(GLfloat), &_screenQuadVertices[3] ); | |
glEnableVertexAttribArray (ATTRIBUTE_BASIC_POSITION); | |
glEnableVertexAttribArray (ATTRIBUTE_BASIC_TEX_COORD); | |
glBindTexture(GL_TEXTURE_2D, _fboTextureID); | |
glDrawElements ( GL_TRIANGLES, 6, GL_UNSIGNED_SHORT, _screenQuadIndices ); | |
glBindTexture(GL_TEXTURE_2D, 0); | |
} |
You might be wondering what all the bindDrawable is about. Well normally in OpenGL you would glBindFrameBuffer(…) to your framebuffer then perform a glBindFramebuffer (…, 0) to reset to the default framebuffer – well that won’t work on OpenGL ES (well you could perhaps if you knew the actual handle). You must use bindDrawable() instead.
Here are some books that helped me greatly
OpenGL® SuperBible: Comprehensive Tutorial and Reference (4th Edition)
i’m so glad you posted this information. I’ve been trying to figure out what was going on for days and was in the process of opening a case with Apple to get some help resolving what was happening to me. I couldn’t figure it out, but now it all makes sense. I couldn’t find anything online including Apple’s online info, Stackoverflow, etc.
Thanks for taking the time to share!
LikeLike
Thanks, anytime. Enjoy the rest of opengl 🙂
LikeLike
hi MickyD
i am working on a GLView record app, i need to get the texture data and record that to a video.
Do i need to create offscreen framebuffer and draw it and get the texture?
and do i need do draw the same thing with main framebuffer to get the texture?
what i knew is that i can bind texture to the main framebuffer and then i can get the texture, is that correct?
i am new to opengles, can you give me more information detailedly? i am thankful for that.
LikeLike
Hi Lee’, from memory you just create an offscreen buffer; bind it; and render to that, you don’t need to render to the main buffer in order to read the texture. This is because on the iPad there is no difference between RAM and GPU RAM – it’s unified which is rather nice. It’s fast too, normally doing this sort of thing on the desktop would be incredibly slow.
LikeLike
i am testing it on my iphone5, but i got a black video, what’s the possible reasons about that?
i have spent hours on this, could you have some time to look at my demo code? i can send it to you.
LikeLike
hi Micky: i put my question on stackoverflow http://stackoverflow.com/questions/24690512/gpuimage-record-opengl-es-scenes-texture-video-is-black, can you give it a look?
LikeLike
I gave you an upvote 🙂 . I put some pointers there too, not sure how useful there are as I don’t know much about the video APIs
LikeLike
Hi Micky,
Can you please tell us the values of
glVertexAttribPointer (ATTRIBUTE_BASIC_POSITION, 3, GL_FLOAT, GL_FALSE, 5 * sizeof(GLfloat), _screenQuadVertices );
glVertexAttribPointer (ATTRIBUTE_BASIC_TEX_COORD, 2, GL_FLOAT, GL_FALSE, 5 * sizeof(GLfloat), &_screenQuadVertices[3] );
glEnableVertexAttribArray (ATTRIBUTE_BASIC_POSITION);
glEnableVertexAttribArray (ATTRIBUTE_BASIC_TEX_COORD);
for a full screen texture of iPhone? I am sorry but i am very new to this. I have created app in which i am doing exactly the same, but the off screen frame buffer is drawing perfectly(i saved as an image) but the same texture is not getting rendered. ican send you the code as well.
LikeLike