Native code for flash

Adobe is just insane.

I'm again having some time and space in my mind to look at the world around me. A huge work related milestone was reached today. Something ended and something starts. It's a really big shame that the things that do succeed in my new position are definitely something I would not be able to share for a long long time with anybody outside of Nokia.

I wrote some time ago about how insane the flash 11 is for 3D support with a very concise representation compared to that of html5 and silverlight. Now, adobe really has done it. Adobe alchemy allows conversion of c/c++ code into low level actionscript bytecode. Secure execution at native speed. Full GDB support, compiles with gcc, supports all the tricks c++ has in its sleeve. Just insane. The bytecode is now running at 40% native, wiht high hopes to get it to 80% native speed before alchemy is launched. You also get finally direct access to converted c++ libraries from AS code. All existing code assets are finally browser compatible.

If you are interested in where online gaming is heading in the future, look no further.

As a demonstration, Epic has converted the >1MLoc unreal engine with alchemy. Watch this whether you are or are not not a coder.


If you are a coder, you need to reserve 45 minutes and watch the video. I am serious. You *have* to watch it.


This is bliss. Triple-a -level games coming to facebook ;). Then push button conversion to adobe air application for your favorite i or android device. I wonder how this will work for windows phone in the future.

N9 on sale in Finland ... and a few other things

I'm fell for it again. Long period of silence. It doesn't mean that things have stood still.

First of all, a very close friend of mine died. Gary Birkett, lcuk to the community, lost his life a few weeks back. For this, for me, world suddenly stopped in its tracks. He was truly an inspiration to us all. I wish all the best to his family while they seek to cope with the tragic loss. To Gary I raise a pint of Guinness. Rest in peace my friend. You will be missed, time and time again.

N9 finally shipped. I was in my earlier post saying that it was a swan song for Nokias Meego efforts, but as faith put it, it is likely the swan song for Meego proper.

It's hard to put into words the feelings we have been having during the late summer/early autumn time. Software has been complete since summer, but some surprise issues had been helding the sales start for some time.

That's business as usual, but not very pleasant ever. We did a few touches here and there all the time until the release, but it was just polish really.

While the agonizing wait was ongoing, software teams worked furiously on making the experience even better. First update release is complete, we will be distributing fruits of it to developers soon and end users a bit further down the road.

Improvements are a legion. This is due to the fact that many teams stopped the sales release work already in spring time and set their sights on the first update release. I counted around 3500 improvements (fixes or features) to be in the release.

Out of interest, I tohttp://www.blogger.com/img/blank.gifok a look at the git activity of meegotouch.
http://meego.gitorious.org/meegotouch/libmeegotouch/commits/Harmattan_1.0
11.8.2011
http://meego.gitorious.org/meegotouch/libmeegotouch/commits/Harmattan_1.1
23.9.2011
http://meego.gitorious.org/meegotouch/libmeegotouch/commits/Harmattan_1.1.1
28.9.2011

Of course, those dates do not mean that the versions were finalized on those days, just that those were the days that last tags of meegotouch were made. For example meegotouch theme for 1.1.1 was update just today, so, it probably isn't ready yet.
http://meego.gitorious.org/meegotouch/meegotouch-theme/commits/Harmattan_1.1.1

Now it's time that we are going full steam ahead with the 1.2 update release, where work is proceeding nicely.

Back to other topics. I have been "grinding" the triple A title games that I have missed, with most recent being the portal, and modern warfare 1&2. After all the hype, I was a bit disappointed about portal. Sure, the mood was pretty OK, but writing was amateurish and the "hidden" marks of the previous lab rats (you?) didn't tell enough story to me. SUmmarising, it was fun, but didn't really pull me in so that I would have to start portal 2 immediately.

That was not the case with call of duty modern warfare. When the very first mission started in random boat in the middle of Atlantic, I was totally hooked in. This was totally immersive game and I really had to drag myself away from the game two times. I didn't even touch the multiplayer after playing it though, just went to steam and bought the second in the trilogy.

Unfortunately, second was this time not as good as the first in the series. It was generally ok, but the events felt very detached from each other. Action was still good and level design was fun to play, but it wasn't such a holistic experience of being there as in the first. If you haven't played those games yet, do yourself a favor and don't start at 10 PM, but 9 PM. Games last only 6-7 hours and they are total adrenaline rush. You won't fall asleep after playing.

I have also been playing with windows 8 a bit. Truly, a leap for Microsoft. Both on the base OS side (it boots so fast that it should default to hibernation rather than standby), as well as UX side. Metro UI is clear, simplified in a good way and beautiful. I'm not a fan of the Metro home screen concept though. I think it was barely adequate for windows phone, but it is too limited for tablet use. Clutter really. However rest of the tablet & phone UI is a jewel. It's so easy to use that even my 2,5 years old daughter is able to power it on, unlock it with the picture password, open metro home and start drawing with the paint application. Not all rosy though, apps start horribly slow, they crash all the time, perform really badly and seem to be generally been given at most a few hours of effort. I hope they are there to set an example that you can do better. Fortunately, there is always the full OS behind that makes the tablet pretty OK even though it's just a developer hacking device.

I was asked recently where I failed during harmattan. I answered that I didn't resist enough the pressure and the temptation of doing the events screen to home. While it serves its purpose well and performs (especially in 1.2) much better than the apps pushing data to it, I think we should have left it out. It's a bit of clutter in the same way as metro home is clutter and cost us a lot of development and management time. Time we might have spent elsewhere better.

But now that I was thinking of windows 8, I think I would revise that. The biggest thing I failed to understand to need a push was replacing backstacking with horizontal paneling.

N9 home is clearly a single layer where you move horizontally between the three different views. On top of home, you have application layer. But with back stacking, applications themselves start to visually have many layers on top of each other. Leaving menus and dialogs aside, I think it really doesn't suit the UI paradigm.

Correct and clinical (like my friend Otso would say) would have been to keep applications in single layer. Lock screen sits on top of applications, applications sit on top of home. All views, move horizontally to show different aspects of the layer. Virtual KB, menus and dialogs are ok to float on top of anything. As back button is not needed anymore, need for toolbar overall diminishes considerably.

Back button really has held the UI design. Swipe would also have been strengtened as a concept by becoming the navigation tool also within the applications. Windows mobile 7 devices have the back as a physical button allowing the UIs themselves to not need a toolbar-alike construct for consumption views, only for productivity views.

I'm kind of signing myself away from the meego also with this blog. I have started already working part time in the smart devices organization (that's windows phones in nokia terms) in forward center. Work that I'm doing now is at the moment revolving around hardware problems now than software. I assume that to change a bit more as we start getting a clear picture on where we want to be going with hardware in the future.

It is very exciting work indeed. We are looking at the devices coming out in 2013 / 2014 now and the tech for that time is something mind boggling. For a long time, it seemed to me that invention was stalling again after apple iphone 4 came out. iphone was revolutionary, but 4 didn't bring anyting big on top of 3/3gs, just nicer looking, more power and a bit more resolution. 4S doesn't seem to be bringing much new either, just better and faster than previous model. Android has been mostly fixing the flaws it has had all the time, and trying to make the platform vaguely performant.

When I'm looking at the next generation of CPU/GPU combinations, I'm almost speechless on what could be done with them. We were stuggling so much with N9 to get the 60 fps possible on the 848x480 display (have you guys noticed hidden 16 pixel rows by the way? <- hackers take a peek and gain a few thousand pixels more) with the limits that we had. These limits are now totally gone. There is no limit for 2.5 D view rendering performance neither on CPU or GPU side.

2013 onwards, industry will make a huge leap in CPU and GPU power and start being close to PS3 & Xbox power levels. It's so fast that even Android can start innovating new things. ;)

This leap of power will usher a new age for mobile devices - if we play our cards right (I mean, as the industry, not only as Nokia). Display technology is advancing at immense rates. Transparent and flexible OLEDs seem to raise so many opportunities that it makes my imagination just tingle of working to set the playground ready for the next innovations on the software front.

If you see issues in n9, don't hesitate to contact me. I'm still well connected to nokia meego and there is a lot of talent working on making it all the time better.

Perhaps it's time to call it a day.

Molehill - Flash 11

Somehow, in the flurry of figthing to get N9 out, I missed Adobes latest big thing.

I have always loved pretty much everything adobe has been pushing out, but this one is just insane.

New Adobe Flash 11 will contain support for 3d acceleration, totally abstracted from opengl/DirectX/OpenGlES or lack of any acceleration.

Guys at Adobe have done outstanding job at abstracting the GPU acceleration to work the same on all platforms (MS, Linux, Osx, Mobile) - yet, you still get to code shaders all the same for all of them.

One of course immediately compares Stage3D to webgl and Silverlight 5. Silverlight 5 looks to me like a nice first try for MS. Webgl, while nice as an idea of pulling it all to browser itself, lacks the compatibility between browsers, and does not even try to protect code copying.

One of the most interesting, and perhaps odd things about the stage3d/molehill, is that adobe somehow resisted trying to crete their own development & design tools for 3d. It would have been a futile effort to do it the whole way from 3d object modelling to animation crafting in a way that pleases all, so they decided to stick with doing just the 3d abstraction layer and do it par none. Having been testing it a bit, I must applaud for job well done.

By not doing the toolset, they have in effect taken all existing adobe 3d engine creators to come onboard as well as converted some of the other engine developers like Unity. While Unity says this is not the end of their dedicated web based player, I don't see any reason why they would keep two competing implementations alive.

For Unity, you code your game with normal c#/javascript/boo, which gets converted to actionscript. These days scripts are mostly running game decision logic, and pretty much all of the heaevy lifting is done on shader programs. So, not a whole lot of perf is lost by sticking completely to actionscript.

For most of the other Stage3D based engines, it's business as usual, use maya/3dmax/whatever makes you tick to do the meshes and place the textures, use the 3d game RAD tool to get it all together and actionscript to control it all.

Let me give you and example of how the code looks for a simple triangle with color slides between the vertices between the three competitors:


WebGl (note that I have dropped shader creation completely -> read from source) source : Source

//Vertexbuffer

    triangleVertexPositionBuffer = gl.createBuffer();

    gl.bindBuffer(gl.ARRAY_BUFFER, triangleVertexPositionBuffer);

    var vertices = [

         0.0,  1.0,  0.0,

        -1.0, -1.0,  0.0,

         1.0, -1.0,  0.0

    ];

    gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(vertices), gl.STATIC_DRAW);

    triangleVertexPositionBuffer.itemSize = 3;

    triangleVertexPositionBuffer.numItems = 3;



    triangleVertexColorBuffer = gl.createBuffer();

    gl.bindBuffer(gl.ARRAY_BUFFER, triangleVertexColorBuffer);

    var colors = [

        1.0, 0.0, 0.0, 1.0,

        0.0, 1.0, 0.0, 1.0,

        0.0, 0.0, 1.0, 1.0

    ];

    gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(colors), gl.STATIC_DRAW);

    triangleVertexColorBuffer.itemSize = 4;

    triangleVertexColorBuffer.numItems = 3;

//Shader creation 

//--> Read from the source, it's far too long to express here

Ms Silverlight 5 Source Full source :

   vertexBuffer = CreateTriangle(); 



// Shader creation: 

   Stream shaderStream = Application.GetResourceStream(new Uri(@"TriangleSample;component/Triangle.vs", UriKind.Relative)).Stream; 

   vertexShader = VertexShader.FromStream(resourceDevice, shaderStream); 

 

   shaderStream = Application.GetResourceStream(new Uri(@"TriangleSample;component/Triangle.ps", UriKind.Relative)).Stream; 

   pixelShader = PixelShader.FromStream(resourceDevice, shaderStream); 



//Vertex buffer

   var vertices = new VertexPositionColor[3]; 

   vertices[0].Position = new Vector3(-1, -1, 0); // left 

   vertices[1].Position = new Vector3(0, 1, 0);   // top 

   vertices[2].Position = new Vector3(1, -1, 0);  // right 

   vertices[0].Color = new Color(255, 0, 0, 255); // red 

   vertices[1].Color = new Color(0, 255, 0, 255); // green 

   vertices[2].Color = new Color(0, 0, 255, 255); // blue 

             

   // create graphics device managed buffer 

   var vb = new VertexBuffer(resourceDevice, VertexPositionColor.VertexDeclaration, 

                vertices.Length, BufferUsage.WriteOnly); 

 

   // copy vertex data to graphics device buffer 

   vb.SetData(0, vertices, 0, vertices.Length, 0); 

Adobe Flash Stage3D Source :

//Shader creation

   context3D.setProgramConstantsFromMatrix(Context3DProgramType.VERTEX, 0, modelMatrix, true );



   var vertexShaderAssembler : AGALMiniAssembler = new AGALMiniAssembler();

   vertexShaderAssembler.assemble( Context3DProgramType.VERTEX,

     "m44 op, va0, vc0 \n" + // 4x4 matrix transform from stream 0 (vertex position) to output clipspace

     "mov v0, va1 \n"    // copy stream 1 (vertex color) to fragment shader

   );



   var fragmentShaderAssembler : AGALMiniAssembler= new AGALMiniAssembler();

   fragmentShaderAssembler.assemble( Context3DProgramType.FRAGMENT,"mov oc, v0);" // output color 

  



//Vertex buffer

   vertexbuffer.uploadFromVector ( Vector.([

     4.-1,-1,0,  255/255,0,0,              // red

     5.0,1,0,    193/255,216/255,47/255,   // green

     6.1,-1,0,   0,164/255,228/255         // blue

     7.]),0, 3 ); // start at offset 0, count 3



   context3D.setVertexBufferAt( 1, vertexbuffer, 3,Context3DVertexBufferFormat.FLOAT_3 );

What I find interesting is that at the same time, Stage3d example code may look a bit more complex than the other, but that's the efficiency and concise presentation talking.

Silverlight stays the most readable of them all, but mostly because shader program is in separate file.

I think this is the new platform that games developers for facebook and mobile will embrace. It will be the basis of new wave of creative coding frameworks, ditching openframeworks and alike.

While also silverlight 5 seems to have a nice home for indie game development for 360+WP7/8, I don't believe it to have as significant impact on gaming as a whole as Stage 3d from adobe will.

I'm in awe.

Now, take a look at how Adobe presents this:


Don't mind about how "very very exciting it is"

If you want a hands on, try the max racer yourself:
http://www.molehillgaming.com/label/stage3d

First install Flash 11 Beta, then open the game.

Respect

Through unimaginable hardships, we managed to pull a rabbit from the hat and get N9 done.
Schedule was just insanely tight, we got blows from all directions: Vanjoki leaving us, Ari leaving us, Elop dropping of meego from strategy, key developers leaving the company when we needed them the most.

Swipe UI, that we are now admiring, required every single application and framework to be rebuilt almost from scratch. All the blows and all the work in the course of 9 months. I never knew how much sweat and tears there needs to be to get this baby out.

What I want to say to the people who beared all that: Respect. You have my utmost respect and admiration. We didn't just do our best, we crafted the software to near perfection. Simplistic UI is easy to imagine, but extremely hard to execute. And execution includes design. Devil is in details and temptations are many. Keeping the boat steady on course needed a lot from you design and product management.

It's nothing short of miracle that the fruit of our labor is now admired by millions.

I am extremely proud of what I am holding in my hand right now. As a user, I'm loving every minute of using it and wouldn't change it for a thing.

All of maemo: I salute you. Job well done. Be proud.

NuvoFre 1.10

NuvoFre 1.10 out with nice summer feel to it.

I'll add it to devel soon.





Direct link: NuvoFre 1.10