Raydium 3D Game Engine

Official forum for everything about Raydium, ManiaDrive, MeMak, ...
It is currently Tue Sep 26, 2017 2:49 pm

All times are UTC




Post new topic Reply to topic  [ 6 posts ] 
Author Message
 Post subject: Texture load speed up
PostPosted: Thu Oct 21, 2010 11:50 am 
Offline

Joined: Sun Oct 09, 2005 10:46 pm
Posts: 759
Hello,

I try to speed up a bit texture loading.

After first look at the code i have two questions:

First one:
texture.c line 314 we use GL_RGB as format to store texture color.

But it seems that GL_BGR exist too. And tga file are in a BGR scheme.

Is it possible to change this, and allow direct load from file to pixel array ? Or there will be enormous drawback every where ?

Second one:
Line 316 and 320. As i understand, even on 3 bpp we use RGBA format. Thats probably why texsize is modified on line 320 (adding 1 bpp).
But if phoneos is defined, does this textsize adjust wont hurt ?

In texture read, if bpp ==3 read part does not store pixel in a 4 bpp alignement. Isn't it problematic ?
Then line 352 texture size calculation is wrong as it doesn't meet malloc requirement if texsize have been corrected (line 320)


If someone can have a look ?

Have a nice day.
Ouille


Top
 Profile  
 
PostPosted: Thu Oct 21, 2010 7:33 pm 
Offline
User avatar

Joined: Sun Mar 16, 2003 2:53 am
Posts: 2590
Location: gnniiiii (Scrat)
Hi !

On specific points:
- GL_BGR can generate a few issues, mainly about portability. I'm also not sure that this format will be the most optimized in GL drivers. On the other hand, the BGR->RGB is probably almost costless, even on low end (embedded included) hardware. We can time it, to be sure of that.
- Yes, we force texture *internal* format to RGBA even when it's not needed. The reason given in the comments is now probably obsolete, ATI's drivers were probably fixed a long time ago, now.
- this GL_RGBA value is the "GPU internal storage mode", so there's no trouble with malloc'ed CPU memory, that is still RGB. The reason why texsize value is updated is that it's used at the end of the function to estimate (roughly) the amount of memory needed for this texture in the hardware, so we take count of this extra byte. To be exact, this incrementation (line 320) should be *inside* the ifndef, not after (it makes the estimation wrong on iPhone target).

On a more global aspect about texture loading speed, I'm pretty sure that optimizing this function, even with a lot of efforts will give almost no results at all. Most of the consumed time is on I/O, since there's a huge amount of data to transfer from the disk to the RAM. The only way, I think, to improve performances ... is to use another (compressed) texture format. I don't see a lot of candidates: JPG, for known reasons is a no-go for me :) , PNG does not compress that much (should make a test, anyway, to compare file sizes of RLE encoded TGAs with PNGs), the only other potential option is the DDS format, that supports some hardware compliant compression formats like S3TC (that Raydium supports too). But it will not be suited for every use (normal maps does not correctly support any type of destructive compression, for instance), and adds a new dependency to the engine

To make long story short, It may be me, but I don't see any easy way to reduce the disk-to-RAM amount of data during a texture load ... Any idea ? :)


Top
 Profile  
 
PostPosted: Thu Oct 21, 2010 8:56 pm 
Offline

Joined: Sun Oct 09, 2005 10:46 pm
Posts: 759
Hello,

I can speed up pixel reading, i can gain 20% on this. The rest is in the tri file read the scanf wich is quite slow.
In fact i think that tri file format is the clue.

I work with some big scene (tri file more than 10Mo with quite big textures), and loading time is quite long, espacialy between to tests.
Vertex coordinate reading consume about 3 secs on 10 secs loading.
And texture affectation is called at each vertex, lots of find function here about 4 sec here.
Texture loading is about 3-4 secs.

I'll perhaps try to use a binary format. What do you prefer, a raydium specific binarie format, or perhaps trying to use an more standard one ? Which one ?

It is not urgent, i initially thought that my code to read pixel was the slowest part, and in fact it is not.

Waiting for suggestions ...

Have a nice day.
Ouille


Top
 Profile  
 
PostPosted: Sat Oct 23, 2010 4:32 pm 
Offline
User avatar

Joined: Sun Mar 16, 2003 2:53 am
Posts: 2590
Location: gnniiiii (Scrat)
yep, "binary tri format" seems ok, but it should probably follows completely current text format structure (same fields, in the same order, ...) to make this easier to write and to maintain, I think.

For compatibility, I would suggest that the file begins with a text line, with specific version numbers (some thing like 100 ["basic"], 101 ["with normals"] and 102 ["animated" ... maybe not interesting currently]) like current files.

We should make sure that it really speeds up loading (more than a few %, I mean), since it's more or less some duplicated code, and it makes tri files less "understandable" to the user, so it have to prove it deserve its presence into the engine :)

Also something to think about how do you convert text-tri files to binary-tri ones ? modler ? blender Python script ? in-game console command ? ...


Top
 Profile  
 
PostPosted: Sun Oct 24, 2010 10:51 am 
Offline

Joined: Sun Oct 09, 2005 10:46 pm
Posts: 759
Amazing !!

For the first time we are quite agree on everything !

My idea was the same for the first textline, for the number beginning at 100, and why not for à .bri extension.

About the format: actually each vertex embend texture information. But each texture information need to be "resolved" for each vertex. I think it's the slowest part in .tri reading.

A think that a chunk with every texture information and a given "id" in the first part of the file allow to do this look up only once.

Secondly vertex and "id" of texture.

On other way can be to order vertex with texture info, speeding lookup : only once for a group of vertex.

BUT, i've just realised that i don't have commit kmz_2_tri work on raydium, so i'll try to finish this first.

See you on an other thread soon ...

Have a nice day.
Ouille.


Top
 Profile  
 
PostPosted: Thu Nov 18, 2010 12:41 pm 
Offline
User avatar

Joined: Sun Mar 16, 2003 2:53 am
Posts: 2590
Location: gnniiiii (Scrat)
I've added very quickly a patch (r1037) to the trunk that allows to display the number of "uncached" calls to raydium_rendering_prepare_texture_unit(). This function is switching the current texture for a specified TMU, and therefore change OpenGL context, witch is a slow operation.

Since we'll be working with ouille on a new way to optimize tri files, this information is important.

ouille: with our test file (parts_demo.tri), it currently reports 7 switch (which is strange). I'll have a look at this.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 6 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group