3D Embedded Web Browser

[TylerH]

Following in Niosop's footsteps, I borrowed his technique of blitting, if you want to call it that, the contents of an unsigned char buffer into a Leadwerks texture handle the contents of a video.

 

I applied it to the Awesomium library, which is a wrapper of Google's Chromium Web Engine, built ontop of Java V8, Web Kit, and some nice HTML renderer, and it supports everything including Flash, HTML5, CSS3, etc. because of the dependence on Webkit.

 

I integrated it into a small, kind of hackish, and horribly unoptimized demo that also handles mouse movement (in 3D) and mouse buttons, however, the mouse XY values skew at obligue angles. So, if anyone knows how to unproject and reproject 3D mouse coordinates to get them relative the camera rotation, that would make this work for all shapes at all angles.

 

Picture is in the gallery.

 

Here is the code:

#include "engine.h"
#include <gl/gl.h>

#include "WebCore.h"

static Awesomium::WebView* webView = 0;
static TBuffer currentbuffer = 0;
static TBuffer testbuffer = 0;
static TTexture testtexture = 0;
static bool drawPage = false;

class MyWebViewListener : public Awesomium::WebViewListener
{
public:
MyWebViewListener()
{
}

void onBeginNavigation(const std::string& url, const std::wstring& frameName)
{
	drawPage = false;
	//std::cout << "Navigating to URL: " << url << std::endl;
}

void onBeginLoading(const std::string& url, const std::wstring& frameName, int statusCode, const std::wstring& mimeType)
{
	//std::cout << "Begining to load URL: " << url;
	//std::cout << "\n\twith status code: " << statusCode;
	//std::wcout << L"\n\twith mime-type: " << mimeType << std::endl;
}

void onFinishLoading()
{
	drawPage = true;
}

void onCallback(const std::string& name, const Awesomium::JSArguments& args)
{
}

void onReceiveTitle(const std::wstring& title, const std::wstring& frameName)
{
}

void onChangeTooltip(const std::wstring& tooltip)
{
}

#if defined(_WIN32)
void onChangeCursor(const HCURSOR& cursor)
{
}
#endif

void onChangeKeyboardFocus(bool isFocused)
{
}

void onChangeTargetURL(const std::string& url)
{
}
};

int nextPow2(int x)
{
	int y;
	for (y=1;y<x;y*=2);
	return y;
}

int main( int argn, char* argv[] )
{
Initialize() ;
RegisterAbstractPath(".");
SetAppTitle( "Leadsomium" ) ;
Graphics( 800, 600 ) ;
AFilter() ;
TFilter() ;

TCamera camera;

TFramework framework=CreateFramework();
TLayer layer = GetFrameworkLayer(0);
camera=GetLayerCamera(layer);

SetSSAO(0);
SetBloom(0);
SetHDR(0);
SetAntialias(1);
SetGodRays(0);

PositionEntity(camera,Vec3(0,0,-3));

TMesh cube = CreateCube(0);
PositionEntity(cube, Vec3(0, 0, 0));
ScaleEntity(cube, Vec3(2,2,2));
//RotateEntity(cube, Vec3(-90,0,0),1);
//TurnEntity(cube, Vec3(0,90,0));
//PointEntity(cube,camera,1);
//PointEntity(cube,camera,2);
//PointEntity(cube,camera,3);

testbuffer = CreateBuffer(1024,1024,BUFFER_COLOR);
testtexture = CreateTexture(1024,1024,TEXTURE_RGBA);
TMaterial testmaterial = LoadMaterial("abstract::web.mat");
TextureFilter(testtexture, TEXFILTER_PIXEL);
SetMaterialTexture(testmaterial, testtexture);

PaintEntity(cube, testmaterial);

Awesomium::WebCore* webCore = new Awesomium::WebCore(Awesomium::LOG_VERBOSE, true, Awesomium::PF_RGBA);

webView = webCore->createWebView(1024, 1024);
webView->loadURL("http://www.leadwerks.com/werkspace");

MyWebViewListener* myListener = new MyWebViewListener();

webView->setListener(myListener);

while( !KeyHit() && !AppTerminate() )
{
	if( !AppSuspended() ) // We are not in focus!
	{

	TurnEntity(cube, Vec3(0,0.025,0));

		//if (isRunning)
		webCore->update();

		if (drawPage)
		{
			unsigned char* buffer = new unsigned char[1024 * 1024 * 4];

			webView->render(buffer, 1024 * 4, 4);

			currentbuffer = CurrentBuffer();
			SetBuffer(testbuffer);

			BindTexture(testtexture);
			glTexImage2D(GL_TEXTURE_2D,0,GL_RGBA,1024,1024,0, GL_RGBA,GL_UNSIGNED_BYTE,buffer);

			SetBuffer(currentbuffer);

			delete buffer;
		}

	// Update timing and world
	UpdateFramework();

	// Render
	RenderFramework();

		TPick p;

		CameraPick(&p, camera, Vec3(MouseX(), MouseY(), 500));

		float mx = 0;
		float my = 0;
		if (p.entity && p.entity == cube)
		{				
		 mx = p.X / EntityScale(p.entity).X;
		 my = p.Y / EntityScale(p.entity).Y;
		 mx = mx + 0.5;
		 my = my + 0.5;
		 my = 1.0 - my;
		 mx = mx * 1024;
		 my = my * 1024;
		 webView->injectMouseMove(mx,my);

		 if (MouseHit(1))
		 {
			 webView->injectMouseDown(Awesomium::LEFT_MOUSE_BTN);
		 }
		 if (MouseHit(2))
		 {
			 webView->injectMouseDown(Awesomium::RIGHT_MOUSE_BTN);
		 }
		 if (MouseHit(3))
		 {
			 webView->injectMouseDown(Awesomium::MIDDLE_MOUSE_BTN);
		 }

		 if (!MouseHit(1))
		 {
			 webView->injectMouseUp(Awesomium::LEFT_MOUSE_BTN);
		 }
		 if (!MouseDown(2))
		 {
			 webView->injectMouseUp(Awesomium::RIGHT_MOUSE_BTN);
		 }
		 if (!MouseDown(3))
		 {
			 webView->injectMouseUp(Awesomium::MIDDLE_MOUSE_BTN);
		 }
		}

		DrawText(0,40, "LX: %f, LY: %f", mx, my);

	// Send to screen
	Flip(0) ;
	}
}

// Done
return Terminate() ;
}

[TylerH]

Just to do some splainin:

 

That web view listener could be much more robust. Right now, I use a hacky static boolean so that we don't try blitting the web view to the texture while it is loading (if you do, it will hang the program), and simply disable rendering while loading, and re-enable it when finished (this is infact what most browsers will do unless streaming the page).

 

There is no keyboard input, becuase it requires you to use some Win32-style structures that I don't know much about.

 

The mouse input is slightly dodgy.

 

This is rendered in realtime, and the FPS is kind of horrid as you an tell. Ways to improve this would include not re-creating the huge char* buffer every frame, utilizing Awesomium's asynchronous (Stream) rendering, using Awesomium's Rendered Rect to determine what parts of the web page have changed and need redrawn, among other things.

[Rekindled Phoenix]

This is wonderful! Will there be a demo in the near future?

[Niosop]

Hell yeah dude! I'll see about optimizing this and adding keyboard control tomorrow. With video and web access that's two more features we can check off in the feature list against Torque3D Add in the node based material editor Josh is working on and we're making real progress. Still need a more robust particle system editor to match Cascade and a physics/joint editor ala PHAT and we'll be approaching the UDK feature set.

[TylerH]

Muahaha. In-game advertising anyone?

[TylerH]

Removing the recreation of the char* buffer every frame and turning on asynchronous rendering only gained me a few frames.

[Josh]

Can you post a compiled demo? This looks neat.

[Niosop]

Here's one, it's just a compiled version of TylerH's code, I just changed the page that's loaded, the resolution and the name of the material/shader. I believe it includes all the files you need that aren't default LeadWerks files.

 

http://206.222.13.171/media/WebDemo.zip

[Niosop]

Doh, forgot that that one also uses DrawImage to do a full screen browser instead of rendering to a cube. Here's a new main.exe that puts it on a cube.

main.zip

[Niosop]

Unless I'm missing something, the mouse picking code will only work if the surface you're clicking is pointing directly at you. In order to pick an arbitrary position you'd need to be able to do one of the following:

 

Figure out the UV coord of the point that was picked. Could probably be done w/ some math that I can't think of at this time of night, or by using a much more tessellated surface so you could use the triangle index to get the UV.

 

Hide the mouse and draw your own cursor onto the texture before it's rendered. This way you could track and inject any mouse movement you wanted.

 

Not sure which would be a better approach.

[wh1sp3r]

i rather have addon for IE to play LE games on web page like unity, lol

[Canardia]

No, embedded internet in game is much better than embedded game in internet.

You could put the internet pages on cubes, and put them in tubes in the game, lots of tubes.

[Niosop]

Or better yet, stack them into a cave....if only we had someone capable of such heroic stacking feats.

[Josh]

Can someone post a demo with all required files?

[Niosop]

Sure, you want it on a cube or a fullscreen texture?

[Josh]

Cube!

[Niosop]

http://206.222.13.171/media/WebWerks.zip

 

Clicking only sends the correct coords when your looking at it face on. F2 will start/stop it rotating and snap it back into the starting position so you can click on stuff. Load up a YouTube video or Flash game, it's really cool.

 

Let me know if anything is missing.

[Marleys Ghost]

Or better yet, stack them into a cave....if only we had someone capable of such heroic stacking feats.

 

 

 

Macklebee ?

[Josh]

Your zip file is missing or corrupt.

[Niosop]

Grr, sorry, here: http://206.222.13.171/media/WebwerksDemo.zip Double checked it this time.

[carlb]

just tryed it but seem to not to be happy with ati

text is very small and getting line across screen when it updates

it almost like memory curuption.

[Josh]

That's awesome, thanks!

 

You can put a computer in your game and have access to the whole web. Or you can look up tips while you are in the game.

[Niosop]

I'm working on a better demo. Got the picking at an angle problem worked out for one sided surfaces which should be most use cases. Seeing about exposing both this and the video player to Lua using Rick's code from earlier. Eventually you should be able to drop a web viewer object in the editor, select a webpage to go to, link it to an entity, optionally select a surface index, and it will render that webpage to the selected object. At least that's my plan if I can figure out some stuff about lua->dll calls.

[Marleys Ghost]

Grr, sorry, here: http://206.222.13.171/media/WebwerksDemo.zip Double checked it this time.

 

 

Cool, thanks for that Niosop and nice work Tyler

[Marleys Ghost]

That's awesome, thanks!

 

You can put a computer in your game and have access to the whole web. Or you can look up tips while you are in the game.

 

 

I like that idea, could add a twist to in game puzzle solving by having to use the net as a resource ... maybe some parential controls too