Jump to content
Search In
  • More options...
Find results that contain...
Find results in...

Video playing example


Recommended Posts

Here's a minimal example of how to play a video on a texture.

 

First download libtheoraplayer. It's what does all the heavy lifting as far as ogg/theora decoding goes. And if you've ever tried to write your own theora decoder you know it is a fair amount of lifting. You'll need to link against the .lib file and have the .dll present in the same directory as your exe.

 

You'll also need an ogg theora video, change the name/path to whatever you use.

 

Here's the C code:

 

#include "engine.h"
#include <gl/gl.h>
#include "TheoraVideoManager.h"
#include "TheoraVideoFrame.h"

int nextPow2(int x)
{
int y;
for (y=1;y<x;y*=2);
return y;
}

int main( int argn, char* argv[] )
{
       Initialize() ;
       RegisterAbstractPath(".");
       SetAppTitle( "Video Test" ) ;
       Graphics( 800, 600) ;
       AFilter() ;
       TFilter() ;

       TCamera camera;

       TFramework framework=CreateFramework();
       TLayer layer = GetFrameworkLayer(0);
       camera=GetLayerCamera(layer);

	SetSSAO(0);
	SetBloom(0);
	SetHDR(0);
	SetAntialias(1);
	SetGodRays(0);

	PositionEntity(camera,Vec3(0,0,-4));

	TMesh cube = CreateCube(0);
	PositionEntity(cube, Vec3(0, 0, 0));

	TheoraVideoClip *clip;
	TheoraVideoManager *mgr;
	TBuffer currentbuffer;

	mgr=new TheoraVideoManager();
	clip=mgr->createVideoClip("bunny.ogg", TH_RGB, 0, 1);
	clip->setAutoRestart(1);

	float w=clip->getWidth(),h=clip->getHeight();
	float tw=nextPow2(w),th=nextPow2(h);

	TBuffer testbuffer = CreateBuffer(tw,th,BUFFER_COLOR);
	TTexture testtexture = CreateTexture(tw,th,TEXTURE_RGB);
	TMaterial testmaterial = LoadMaterial("abstract::video.mat");
	TextureFilter(testtexture, TEXFILTER_PIXEL);
	SetMaterialTexture(testmaterial, testtexture);

	PaintEntity(cube, testmaterial);

	unsigned long time=GetTickCount();
	unsigned long t=time;

       while( !KeyHit() && !AppTerminate() )
       {
		if( !AppSuspended() ) // We are not in focus!
           {

			TurnEntity(cube, Vec3(0.1*AppSpeed(),0.1*AppSpeed(),0.1*AppSpeed()));
			TheoraVideoFrame* f=clip->getNextFrame();

			if(f) 
			{
				currentbuffer = CurrentBuffer();
				SetBuffer(testbuffer);

				BindTexture(testtexture);
				glTexImage2D(GL_TEXTURE_2D,0,GL_RGB,w-1,h-1,0, GL_RGB,GL_UNSIGNED_BYTE,f->getBuffer());
				clip->popFrame();

				SetBuffer(currentbuffer);
			}

			t=GetTickCount();
			float diff=(t-time)/1000.0f;
			if (diff > 0.25f)
				diff=0.05f; // prevent spikes (usually happen on app load)
			mgr->update(diff);
			time=t;

			// Update timing and world
			UpdateFramework();

               // Render
               RenderFramework();

			// Send to screen
			Flip(0) ;
		}
       }

       // Done
       return Terminate() ;
}

 

The video.mat file is:

 

blend=0
depthmask=1
depthtest=1
overlay=0
zsort=0
cullface=1
castshadows=1
specular=1.00000000
bumpscale=1.00000000
gloss=0.500000000
shader="abstract::mesh.vert","abstract::mesh_diffuse_fullbright2.frag"
shadowshader="abstract::mesh_shadow.vert",""

 

The mesh_diffuse_fullbright2.frag shader is the same as the mesh_diffuse_fullbright.frag shader but without the bloom:

 

#define LW_DIFFUSE texture0
#define LW_FULLBRIGHT

include "mesh.frag"

 

Your can use any material/shader combo that you want for different effects.

 

Some possible improvements:

 

I'm pretty sure the SetBuffer commands really aren't needed, but something that SetBuffer does IS required. I just don't know all the openGL commands that SetBuffer issues, so not sure which ones are required. Probably just resetting some state or something. It doesn't seem to hurt performance much using SetBuffer so I'm not too worried about it, but it might eat up additional VRAM having it sitting around when it's not needed.

 

libtheoraplayer uses a separate thread to do the video decoding and YUV->RGB conversion so the amount of slowdown you see when playing a video will depend on if you have a multi-core machine. Typically the YUV->RGB conversion process takes longer than the actual video decoding, so for slower or single core machines requesting the data in YUV and using a shader to do the YUV->RGB conversion could result in a significant speedup. I'll take a look at doing this if there's any demand. On my machine I actually see no framerate drop at all when playing a video as opposed to just rendering the spinning cube.

 

I'm using GetTickCount instead of AppTime because it seems to offer higher resolution timing, but I haven't tried AppTime since I did some changes, so it might work just as well now.

 

Need to get lower level access to the sound data buffers that LeadWerks uses so that we can reliably sync the video and audio data. It seems that many libraries use the audio system to set the timing and base the video position off of where the audio system is in the audio buffer. Josh, any recommendations?

 

Big thanks to Josh for pointing out the BindTexture command, it's what made updating the texture at a reasonable speed possible.

 

As always, comments and suggestions are welcome.

  • Upvote 1
Link to post
Share on other sites

glTexImage2D(GL_TEXTURE_2D,0,GL_RGB,w-1,h-1,0, GL_RGB,GL_UNSIGNED_BYTE,f->getBuffer());

 

 

That looks to be the only gl command you are calling. What exactly is that doing and was there no LE way of doing it? I'd like to try and get a lua object around this but don't think I can use the gl commands in lua.

Link to post
Share on other sites

You can do a search for ogg sample video or convert one, or download the full version of Big Buck Bunny in Ogg Theora format.

 

glTexImage2D(GL_TEXTURE_2D,0,GL_RGB,w-1,h-1,0, GL_RGB,GL_UNSIGNED_BYTE,f->getBuffer());

 

 

That looks to be the only gl command you are calling. What exactly is that doing and was there no LE way of doing it? I'd like to try and get a lua object around this but don't think I can use the gl commands in lua.

 

It takes the decoded RGB from a libtheoraplayer buffer and copies it over to memory referenced by the LE texture. The BindTexture command seems to do the same as glBindTexture, with maybe some additional housekeeping. There's no way to do it via pure LE API commands.

Link to post
Share on other sites

I will work on modifying this for you to not use the SetBuffer commands, but it WILL introduce a few more direct OpenGL calls, but speed is the key here, not the comamnd count.

 

@Rick: There is a set of OpenGL scripts for Lua that expose it. I believe Lumooja tested them out before and verified they worked, but I am not 100% sure.

Link to post
Share on other sites

Ok, I tested loading it in passing TH_YUV to use the default YUV encoding, and convert it to RGB colorspace in the mesh.frag shader.

 

I would like to note that I got 530 FPS using the TH_RGB with no conversion via shader code, and 500 or lower FPS using the conversion.

 

When you think about it, the mesh.frag shader is going to run much slower because of how large the video resolution is, etc. So it becomes a factor that depends on a lot of variables.

 

Using the shader-based approach really seems like overkill, when with only 1 worker thread LibTheoraPlayer seems to load right into RGB pretty darn fast.

Link to post
Share on other sites

Yup, the only time a shader based approach would make sense is when running on a single core machine where the decoding thread would steal cycles from the engine, then you might want to offload that to the video card. But that's just a guess. Anyone have a single core machine around to test with?

Link to post
Share on other sites
  • 4 months later...

Thank you for uploading this, you are a life (and Project Grade) saver. Is this library usable for commercial games for us to make and sell, in case we ever decide we want to pursue making a commercial version of our game in the future?

Link to post
Share on other sites
  • 1 year later...

Here's a minimal example of how to play a video on a texture.

 

thanks for sharing your code Niosop!

I joined this conversation a little bit late but still I want to ask a question:

why did you define the "testbuffer" buffer?

it seems that you are using it just before calling glTexImage2D() and nowhere else..

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...