Raydium 3D Game Engine

Official forum for everything about Raydium, ManiaDrive, MeMak, ...
It is currently Sun Sep 24, 2017 8:34 am

All times are UTC




Post new topic Reply to topic  [ 23 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Package system (rev 887)
PostPosted: Fri Aug 28, 2009 2:20 pm 
Offline
User avatar

Joined: Sun Mar 16, 2003 2:53 am
Posts: 2590
Location: gnniiiii (Scrat)
You may have seen in the SVN log that I've just added a packaging system.

The idea is very simple: create a ZIP file, add some files in it, and register it in your application. Raydium will then be able to find/use files in ZIP with no particular modification to the application. This feature will probably simplify a lot of things (application testing, deployment, ...)

Off course, you can have many packages registered at the same time. There's also a cache system to make all this quicker as possible. You can have a look at raydium/headers/path.h, the doc is up-to-date.

But !

To write this feature, I needed a new feature from PHP: ZipArchive, so it requires a few changes to SDKs:
- Linux: you must rebuild PHP and Raydium: ./configure --force-php-install && make clean && make
- win32: php.ini should be copied in the bin/ directory, and a the php_zip.dll should be added as well. With the latest SDK, the PHP release is 5.2.9, so the DLL is available here: http://www.php.net/get/php-5.2.9-2-Win3 ... m/a/mirror
- OSX / iPhone: PHP should be compiled with the --enable-zip switch (nothing else needed ?)

To ouille and st: can you update SDKs, guys ?

Note to vicente: this commit also allows "Path API" to use subdirectories, so it will probably help you for the Cal3D animation system, since I've seen some tricky things about paths in the patch. Anyway, I think it's pretty interesting to move from the "data/anim" directory to a package system (one animated model = one package), it will probably simplify your work, and make things waaaay better for users.

Very simple test for this feature with text files (of course, it works to all files: textures, sounds, ...):
Code:
// 'space' key
if(raydium_key_last==1000+' ')
   {
   FILE *fp;
   char data[1024];

   raydium_path_package_register("test.zip");

   // print test file content
   //fp=raydium_file_fopen("zip_flag.txt","rt");
   fp=raydium_file_fopen("zip_subdir/zip_flagsub.txt","rt");
   if(fp)
      {
      raydium_file_binary_fgets(data,1000,fp);
      fclose(fp);
      raydium_log("zip_flag.txt: %s",data);
      }
   else
      raydium_log("failed !");
   }

The archive test.zip is available on official R3S servers, so it will be downloaded.

Please, do not hesitate to comment this commit, I really want to know what you're thinking of this feature (easy to understand or not ? what's wrong ? what good ? what can be changed ?)


Top
 Profile  
 
PostPosted: Fri Aug 28, 2009 3:29 pm 
Offline
User avatar

Joined: Thu Sep 29, 2005 2:59 pm
Posts: 828
Nice (and fast)!
Go to check it and try an anim patch adapted.
I'll post resutls.


Top
 Profile  
 
PostPosted: Fri Aug 28, 2009 4:00 pm 
Offline
User avatar

Joined: Thu Sep 29, 2005 2:59 pm
Posts: 828
Well, tested and it worked directly, impresive. (I'll do the new commit soon)

One comment. When we use a package it's extracted in .raydium/packages/. However it remains there after closing the application.
Is that reasonable? Or it should be erased after clossing app?


Top
 Profile  
 
PostPosted: Fri Aug 28, 2009 4:38 pm 
Offline

Joined: Tue Jul 08, 2008 2:37 am
Posts: 181
Impressive! I love this feature!

I'm currently building PHP with the new build options. After testing it locally I'll prepare it for all architectures and update the SDKs.
Perhaps I'll wait with the Raydium SDKs update release until my test phase with Mac OS X 10.6 aka Snow Leopard and iPhone OS 3.x, but not really sure about it, this could talk some time and I want the SDKs every time to be useable at the newest state of the engine.

Your code snippet is very useful to quickly test this new feature.
Raydium wrote:
Raydium: Using repositories to get 'test.zip' file
Raydium: OK (http://fastrepo.raydium.org/)
Raydium:
Fatal error: Class 'ZipArchive' not found in /Library/Raydium/rayphp/zip_extract.php on line 8

Raydium: Using repositories to get 'zip_flagsub.txt' file
Raydium: FAILED: file not found (http://fastrepo.raydium.org/)
Raydium: FAILED: file not found (http://repository.raydium.org/)
Raydium: No valid repository found for this file, aborting.
Raydium: failed !
IMHO the positive part is that everything runs fine also without the ZIP support, except you cannot use this feature and I'm very eager what is the output when PHP is ready to manage zip archives. :D

Thanks to Xfennec for this great feature, this reminds me to the Quake like package system, and we are also able to give the packages our own file extensions, that'll be very useful in general!


Top
 Profile  
 
PostPosted: Fri Aug 28, 2009 5:01 pm 
Offline
User avatar

Joined: Sun Mar 16, 2003 2:53 am
Posts: 2590
Location: gnniiiii (Scrat)
Thanks for positive feedbacks :)

Short comments :
Quote:
When we use a package it's extracted in .raydium/packages/. However it remains there after closing the application.
Is that reasonable? Or it should be erased after clossing app?

When the application will be launched again, it will probably register the same packages (or some of them a least), and there's no need to extract again the ZIP file (it can be a quite slow task). It's the famous cache of this feature :) (of course, cache is erased if the ZIP file is modified). The only cost is disk space.

Quote:
[..] this reminds me to the Quake like package system, and we are also able to give the packages our own file extensions, that'll be very useful in general!

Id Software PAK files also came to my mind when I wrote this :) The point about custom extension is quite important, I think. What about a specific extension for animated models, for instance ?


Top
 Profile  
 
PostPosted: Fri Aug 28, 2009 5:14 pm 
Offline

Joined: Mon Aug 25, 2008 1:08 pm
Posts: 67
why not a real test with maniadrive? :) or test to make like q3based games: pack the maps :)


Top
 Profile  
 
PostPosted: Fri Aug 28, 2009 8:05 pm 
Offline

Joined: Tue Jul 08, 2008 2:37 am
Posts: 181
Apple related SDKs are now updated!

They include the new PHP dependency files based on the new build options:
Info.txt wrote:
PHP 5.2.6 http://php.net/
Build options: --disable-shared --enable-embed=static --with-zlib --enable-ftp --enable-static=zlib --with-curl --disable-simplexml --disable-xmlreader --disable-xmlwriter --enable-soap --without-iconv --enable-zip

The read me file of each SDK has got a new change log entry:
README.txt wrote:
...
R888:
- Updated the PHP dependency to support ZIP archives.
- Updated to the latest Raydium Subversion revision R888.
...

The package system works on every architecture as expected. Again this is a really great feature and I'm very happy to use it in the future. First I thought about a possible problem, two packages with the same structure, which file will be used by the engine, then I saw that the extracted path also includes the archive filename including the extension. :twisted:

I'll delete the old SDK files from the server, when I've downloaded and double checked the current ones. :)
P.S.: The SDK update "mechanism" works perfect for me.


Top
 Profile  
 
PostPosted: Fri Aug 28, 2009 9:22 pm 
Offline
User avatar

Joined: Sun Mar 16, 2003 2:53 am
Posts: 2590
Location: gnniiiii (Scrat)
Great, thanks !

About a conflict with two files with the same name in two different packages (not really your example), the package system will prefer the first registered package (Path API behaviour, in facts). It's not perfect, but at least, it's stable ;)


Top
 Profile  
 
PostPosted: Sat Aug 29, 2009 11:35 am 
Offline
User avatar

Joined: Thu Sep 29, 2005 2:59 pm
Posts: 828
Maybe this behaviour (take into account the first registered package) could be reversed.
If the system uses the last packages(intead of the first ones) it would be like using a regular filesystem, ie. when you "copy"(=load package) a folder it will overwrite the previous content. This could be usefull for streaming and that kind of things.


Top
 Profile  
 
PostPosted: Sat Aug 29, 2009 11:55 am 
Offline

Joined: Tue Jul 08, 2008 2:37 am
Posts: 181
vicente wrote:
Maybe this behaviour could be reversed.
I totally agree with Vicente, when it's easily possible and quickly done, we should change it.
Looking at Quake again, it's also working the same. You can have several packages, e. g. pak0.pkg, pak1.pkg, ..., you can "overwrite" files internally with the last package loaded.


Top
 Profile  
 
PostPosted: Sat Aug 29, 2009 12:08 pm 
Offline
User avatar

Joined: Sun Mar 16, 2003 2:53 am
Posts: 2590
Location: gnniiiii (Scrat)
Interesting. But it's not easy, because it mean I must change the Path API behaviour: when you add something to a PATH environment variable, first directories are always used first. I'll think about it.


Top
 Profile  
 
PostPosted: Sat Aug 29, 2009 6:21 pm 
Offline
User avatar

Joined: Thu Sep 29, 2005 2:59 pm
Posts: 828
Idea for this:
The packages can be stored (theirs paths) into a different list. And when a path_resolv is called, then we add the reversed list of packages after the common paths.


Top
 Profile  
 
PostPosted: Thu Sep 03, 2009 11:58 am 
Offline

Joined: Tue Jul 08, 2008 2:37 am
Posts: 181
I got some compiling trouble with kinghill2_server.c, mania_server.c and tests.c:
Compiler wrote:
In file included from raydium/rayphp.c:11,
from raydium/index.c:108,
from mania_server.c:4:
raydium/headers/rayphp.h:53: error: conflicting types for 'raydium_rayphp_zip_extract'
raydium/path.c:414: error: previous implicit declaration of 'raydium_rayphp_zip_extract' was here
Seems that problem occurs when RAYDIUM_NETWORK_ONLY was defined.


Top
 Profile  
 
PostPosted: Thu Sep 03, 2009 4:36 pm 
Offline
User avatar

Joined: Sun Mar 16, 2003 2:53 am
Posts: 2590
Location: gnniiiii (Scrat)
Fixed with rev 894. Thanks !


Top
 Profile  
 
PostPosted: Fri Oct 02, 2009 9:21 pm 
Offline

Joined: Sun Oct 09, 2005 10:46 pm
Posts: 759
Hello,

Done in last win32 sdk (896) and tested !!

Is there an easy way to get all media file in binarie directory at runtine (help to prepare deploy of an application).
Retreiving file from either r3s or home directory ?

Is there a magic flag for this ?

Have a nice day
Ouille


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 23 posts ]  Go to page 1, 2  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 2 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group