Jump to content

Video playing example


Niosop
 Share

Recommended Posts

Here's a minimal example of how to play a video on a texture.

 

First download libtheoraplayer. It's what does all the heavy lifting as far as ogg/theora decoding goes. And if you've ever tried to write your own theora decoder you know it is a fair amount of lifting. You'll need to link against the .lib file and have the .dll present in the same directory as your exe.

 

You'll also need an ogg theora video, change the name/path to whatever you use.

 

Here's the C code:

 

#include "engine.h"
#include <gl/gl.h>
#include "TheoraVideoManager.h"
#include "TheoraVideoFrame.h"

int nextPow2(int x)
{
int y;
for (y=1;y<x;y*=2);
return y;
}

int main( int argn, char* argv[] )
{
       Initialize() ;
       RegisterAbstractPath(".");
       SetAppTitle( "Video Test" ) ;
       Graphics( 800, 600) ;
       AFilter() ;
       TFilter() ;

       TCamera camera;

       TFramework framework=CreateFramework();
       TLayer layer = GetFrameworkLayer(0);
       camera=GetLayerCamera(layer);

	SetSSAO(0);
	SetBloom(0);
	SetHDR(0);
	SetAntialias(1);
	SetGodRays(0);

	PositionEntity(camera,Vec3(0,0,-4));

	TMesh cube = CreateCube(0);
	PositionEntity(cube, Vec3(0, 0, 0));

	TheoraVideoClip *clip;
	TheoraVideoManager *mgr;
	TBuffer currentbuffer;

	mgr=new TheoraVideoManager();
	clip=mgr->createVideoClip("bunny.ogg", TH_RGB, 0, 1);
	clip->setAutoRestart(1);

	float w=clip->getWidth(),h=clip->getHeight();
	float tw=nextPow2(w),th=nextPow2(h);

	TBuffer testbuffer = CreateBuffer(tw,th,BUFFER_COLOR);
	TTexture testtexture = CreateTexture(tw,th,TEXTURE_RGB);
	TMaterial testmaterial = LoadMaterial("abstract::video.mat");
	TextureFilter(testtexture, TEXFILTER_PIXEL);
	SetMaterialTexture(testmaterial, testtexture);

	PaintEntity(cube, testmaterial);

	unsigned long time=GetTickCount();
	unsigned long t=time;

       while( !KeyHit() && !AppTerminate() )
       {
		if( !AppSuspended() ) // We are not in focus!
           {

			TurnEntity(cube, Vec3(0.1*AppSpeed(),0.1*AppSpeed(),0.1*AppSpeed()));
			TheoraVideoFrame* f=clip->getNextFrame();

			if(f) 
			{
				currentbuffer = CurrentBuffer();
				SetBuffer(testbuffer);

				BindTexture(testtexture);
				glTexImage2D(GL_TEXTURE_2D,0,GL_RGB,w-1,h-1,0, GL_RGB,GL_UNSIGNED_BYTE,f->getBuffer());
				clip->popFrame();

				SetBuffer(currentbuffer);
			}

			t=GetTickCount();
			float diff=(t-time)/1000.0f;
			if (diff > 0.25f)
				diff=0.05f; // prevent spikes (usually happen on app load)
			mgr->update(diff);
			time=t;

			// Update timing and world
			UpdateFramework();

               // Render
               RenderFramework();

			// Send to screen
			Flip(0) ;
		}
       }

       // Done
       return Terminate() ;
}

 

The video.mat file is:

 

blend=0
depthmask=1
depthtest=1
overlay=0
zsort=0
cullface=1
castshadows=1
specular=1.00000000
bumpscale=1.00000000
gloss=0.500000000
shader="abstract::mesh.vert","abstract::mesh_diffuse_fullbright2.frag"
shadowshader="abstract::mesh_shadow.vert",""

 

The mesh_diffuse_fullbright2.frag shader is the same as the mesh_diffuse_fullbright.frag shader but without the bloom:

 

#define LW_DIFFUSE texture0
#define LW_FULLBRIGHT

include "mesh.frag"

 

Your can use any material/shader combo that you want for different effects.

 

Some possible improvements:

 

I'm pretty sure the SetBuffer commands really aren't needed, but something that SetBuffer does IS required. I just don't know all the openGL commands that SetBuffer issues, so not sure which ones are required. Probably just resetting some state or something. It doesn't seem to hurt performance much using SetBuffer so I'm not too worried about it, but it might eat up additional VRAM having it sitting around when it's not needed.

 

libtheoraplayer uses a separate thread to do the video decoding and YUV->RGB conversion so the amount of slowdown you see when playing a video will depend on if you have a multi-core machine. Typically the YUV->RGB conversion process takes longer than the actual video decoding, so for slower or single core machines requesting the data in YUV and using a shader to do the YUV->RGB conversion could result in a significant speedup. I'll take a look at doing this if there's any demand. On my machine I actually see no framerate drop at all when playing a video as opposed to just rendering the spinning cube.

 

I'm using GetTickCount instead of AppTime because it seems to offer higher resolution timing, but I haven't tried AppTime since I did some changes, so it might work just as well now.

 

Need to get lower level access to the sound data buffers that LeadWerks uses so that we can reliably sync the video and audio data. It seems that many libraries use the audio system to set the timing and base the video position off of where the audio system is in the audio buffer. Josh, any recommendations?

 

Big thanks to Josh for pointing out the BindTexture command, it's what made updating the texture at a reasonable speed possible.

 

As always, comments and suggestions are welcome.

  • Upvote 1

Windows 7 x64 - Q6700 @ 2.66GHz - 4GB RAM - 8800 GTX

ZBrush - Blender

Link to comment
Share on other sites

Excellent and very generous of you to share this :) I will have a play with that now .. I think I have the libtheoraplayer somewhere on the HDD's .. Thank you.

AMD Bulldozer FX-4 Quad Core 4100 Black Edition

2 x 4GB DDR3 1333Mhz Memory

Gigabyte GeForce GTX 550 Ti OC 1024MB GDDR5

Windows 7 Home 64 bit

 

BlitzMax 1.50 • Lua 5.1 MaxGUI 1.41 • UU3D Pro • MessiahStudio Pro • Silo Pro

3D Coat • ShaderMap Pro • Hexagon 2 • Photoshop, Gimp & Paint.NET

 

LE 2.5/3.4 • Skyline UE4 • CE3 SDK • Unity 5 • Esenthel Engine 2.0

 

Marleys Ghost's YouTube Channel Marleys Ghost's Blog

 

"I used to be alive like you .... then I took an arrow to the head"

Link to comment
Share on other sites

If not it's available at http://libtheoraplayer.sourceforge.net/.

 

 

Thanks for that Nio, its probably the "quicker" otpion than hunting though the mess that is my storage drive :)

AMD Bulldozer FX-4 Quad Core 4100 Black Edition

2 x 4GB DDR3 1333Mhz Memory

Gigabyte GeForce GTX 550 Ti OC 1024MB GDDR5

Windows 7 Home 64 bit

 

BlitzMax 1.50 • Lua 5.1 MaxGUI 1.41 • UU3D Pro • MessiahStudio Pro • Silo Pro

3D Coat • ShaderMap Pro • Hexagon 2 • Photoshop, Gimp & Paint.NET

 

LE 2.5/3.4 • Skyline UE4 • CE3 SDK • Unity 5 • Esenthel Engine 2.0

 

Marleys Ghost's YouTube Channel Marleys Ghost's Blog

 

"I used to be alive like you .... then I took an arrow to the head"

Link to comment
Share on other sites

glTexImage2D(GL_TEXTURE_2D,0,GL_RGB,w-1,h-1,0, GL_RGB,GL_UNSIGNED_BYTE,f->getBuffer());

 

 

That looks to be the only gl command you are calling. What exactly is that doing and was there no LE way of doing it? I'd like to try and get a lua object around this but don't think I can use the gl commands in lua.

Link to comment
Share on other sites

You can probably convert any video with the free NCH Video Converter ( http://www.nchsoftware.com/prism/index.html ), and you can download any video from YouTube.

Ryzen 9 RX 6800M ■ 16GB XF8 Windows 11 ■
Ultra ■ LE 2.53DWS 5.6  Reaper ■ C/C++ C# ■ Fortran 2008 ■ Story ■
■ Homepage: https://canardia.com ■

Link to comment
Share on other sites

You can do a search for ogg sample video or convert one, or download the full version of Big Buck Bunny in Ogg Theora format.

 

glTexImage2D(GL_TEXTURE_2D,0,GL_RGB,w-1,h-1,0, GL_RGB,GL_UNSIGNED_BYTE,f->getBuffer());

 

 

That looks to be the only gl command you are calling. What exactly is that doing and was there no LE way of doing it? I'd like to try and get a lua object around this but don't think I can use the gl commands in lua.

 

It takes the decoded RGB from a libtheoraplayer buffer and copies it over to memory referenced by the LE texture. The BindTexture command seems to do the same as glBindTexture, with maybe some additional housekeeping. There's no way to do it via pure LE API commands.

Windows 7 x64 - Q6700 @ 2.66GHz - 4GB RAM - 8800 GTX

ZBrush - Blender

Link to comment
Share on other sites

I've been blitting animated image sequences to planes for a while but this will make life much easier. Thanks for sharing the code.

Intel Core i5 2.66 GHz, Asus P7P55D, 8Gb DDR3 RAM, GTX460 1Gb DDR5, Windows 7 (x64), LE Editor, GMax, 3DWS, UU3D Pro, Texture Maker Pro, Shader Map Pro. Development language: C/C++

Link to comment
Share on other sites

I will work on modifying this for you to not use the SetBuffer commands, but it WILL introduce a few more direct OpenGL calls, but speed is the key here, not the comamnd count.

 

@Rick: There is a set of OpenGL scripts for Lua that expose it. I believe Lumooja tested them out before and verified they worked, but I am not 100% sure.

52t__nvidia.png nVidia 530M cpu.gif Intel Core i7 - 2.3Ghz 114229_30245_16_hardware_memory_ram_icon.png 8GB DDR3 RAM Windows7_Start.gif Windows 7 Ultimate (64x)

-----

IconVisualStudio16.png Visual Studio 2010 Ultimate google-Chrome.png Google Chrome PhotoshopLinkIndicator.png Creative Suite 5 icon28.gif FL Studio 10 MicrosoftOfficeLive.png Office 15

-----

csharp.png Expert cpp.png Professional lua_icon.png Expert BMX Programmer

-----

i-windows-live-messenger-2009.pngskype-icon16.pngaim_online.pnggmail.pngicon_48x48_prism-facebook.pngtunein-web.pngyahoo.giftwitter16.png

Link to comment
Share on other sites

Ok, I tested loading it in passing TH_YUV to use the default YUV encoding, and convert it to RGB colorspace in the mesh.frag shader.

 

I would like to note that I got 530 FPS using the TH_RGB with no conversion via shader code, and 500 or lower FPS using the conversion.

 

When you think about it, the mesh.frag shader is going to run much slower because of how large the video resolution is, etc. So it becomes a factor that depends on a lot of variables.

 

Using the shader-based approach really seems like overkill, when with only 1 worker thread LibTheoraPlayer seems to load right into RGB pretty darn fast.

52t__nvidia.png nVidia 530M cpu.gif Intel Core i7 - 2.3Ghz 114229_30245_16_hardware_memory_ram_icon.png 8GB DDR3 RAM Windows7_Start.gif Windows 7 Ultimate (64x)

-----

IconVisualStudio16.png Visual Studio 2010 Ultimate google-Chrome.png Google Chrome PhotoshopLinkIndicator.png Creative Suite 5 icon28.gif FL Studio 10 MicrosoftOfficeLive.png Office 15

-----

csharp.png Expert cpp.png Professional lua_icon.png Expert BMX Programmer

-----

i-windows-live-messenger-2009.pngskype-icon16.pngaim_online.pnggmail.pngicon_48x48_prism-facebook.pngtunein-web.pngyahoo.giftwitter16.png

Link to comment
Share on other sites

Yup, the only time a shader based approach would make sense is when running on a single core machine where the decoding thread would steal cycles from the engine, then you might want to offload that to the video card. But that's just a guess. Anyone have a single core machine around to test with?

Windows 7 x64 - Q6700 @ 2.66GHz - 4GB RAM - 8800 GTX

ZBrush - Blender

Link to comment
Share on other sites

  • 4 months later...

Thank you for uploading this, you are a life (and Project Grade) saver. Is this library usable for commercial games for us to make and sell, in case we ever decide we want to pursue making a commercial version of our game in the future?

simpleSigPNG.png

 

Programmer/Engineer/Student

www.reikumar.com

 

2.6 GHz Intel Core Duo - nVidia GeForce 8600 GT - Windows 7 64-bit - 4 Gigs RAM

C++ - Visual Studio Express - Dark GDK - Leadwerks SDK

Link to comment
Share on other sites

  • 1 year later...

Here's a minimal example of how to play a video on a texture.

 

thanks for sharing your code Niosop!

I joined this conversation a little bit late but still I want to ask a question:

why did you define the "testbuffer" buffer?

it seems that you are using it just before calling glTexImage2D() and nowhere else..

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

×
×
  • Create New...