Raydium 3D Game Engine

Official forum for everything about Raydium, ManiaDrive, MeMak, ...
It is currently Thu Mar 28, 2024 11:41 am

All times are UTC




Post new topic Reply to topic  [ 31 posts ]  Go to page 1, 2, 3  Next
Author Message
PostPosted: Mon Apr 10, 2006 7:45 pm 
Offline

Joined: Fri Mar 17, 2006 12:41 pm
Posts: 46
Location: Gif/Yvette, France
I tried to sum up the way a Raydium game compiles now. We have several building scripts:
-- comp.sh seems obsolete to me since it does not includes ODE support.
-- ocomp.sh builds the game and raydium statically, linking with everything else dynamically.
-- odyncomp.sh makes Raydium as a lib, then compile the game and link to it ; but the libraydium stays in the main directory and then there is a hack with LD_LIBRARY_PATH to link it.
-- and ostatcomp.sh tries to compile everything in a single binary, but it has yet to link with openGL libs dynamically. I don't see the interest of that method.

Let's have a look at the "local needs" of raydium :
-- ODE is downloaded and included locally. In my distribution (gentoo) it comes as a package : why not use it as such, and link to it from raydium ?
-- Pretty much the same thing with PHP, I didn't look into the internals of the RayPHP magic, but I think it can be done without the need of a local PHP dir.

Look at the base directory. Pretty messy, huh ? I think it would be nice to clearly distinguish games (ManiaDrive and any wonderful games to come) and tests from the library itself.

What do I suggest ? I think it would be nice to :
-- get rid of "local installations" and use system-wide libraries.
-- Use dynamic linking everywhere, and modify the makefile to add a "make install" target, allowing the install of libraydium.so in the /usr/lib/ directory.
-- That way games would compile only against libraydium.
-- Games (or tests) could be in a subdirectory or another repository, making a clear dinstinction with Raydium itself.

I know these ideas have several drawbacks, and I probably misunderstood some parts (I'm pretty new to the project), but I believe the project could interest more people if the boundaries were clear between Raydium the library and the Raydium-based games.

I posted this in english to allow everyone to comment.

_________________
Phyce


Top
 Profile  
 
 Post subject:
PostPosted: Tue Apr 11, 2006 9:02 pm 
Offline
User avatar

Joined: Sun Mar 16, 2003 2:53 am
Posts: 2591
Location: gnniiiii (Scrat)
-- comp.sh seems obsolete to me since it does not includes ODE support.
Some apps, such as raydium_modler.c and willou.c do not use ODE. For a faster compilation, comp.sh is needed.

-- ocomp.sh builds the game and raydium statically, linking with everything else dynamically.
Right.

-- odyncomp.sh makes Raydium as a lib, then compile the game and link to it ; but the libraydium stays in the main directory and then there is a hack with LD_LIBRARY_PATH to link it.
Right, but LD_LIBRARY_PATH is all but a hack ! :) See ld.so(8) if needed. See below for my point of vue on this (unique Raydium installation for the whole system seems wrong for me).

-- and ostatcomp.sh tries to compile everything in a single binary, but it has yet to link with openGL libs dynamically. I don't see the interest of that method.
This script is a must ! That's the only way for us to give Linux users a static version (public release) of games and apps for people that don't want (or can't) compile the whole thing. About 95% of users of our last game (NewSkyDiver) were using static version ! :)
If libGL and libGLU are linked dynamically, it's because they're in facts graphic drivers :) Same problem with asound (ALSA) and thread lib (that's not the same on every Linux system). Again, if we must keep only one of theses "*comp*sh" scripts, ostatcomp.sh is the one.

About local needs:
- libODE isn't provided on every distribution, and when it's the case, it's almost always a "wrong" version : too old (last official release of ODE [0.5] is almost 2 years old !) or the lastest SVN version with unstable sphere/trimesh and so on ... (for example, XMoto, a game using "system wide" libODE lib have a lot of "compilation problem with ODE" like threads in its forum, and is not using advanced features such as trimesh like us... ). The version that's downloaded by Raydium's configure script from CQFD Corp. webserver is known to work perfectly for our needs.

- PHP is (sadly) not provided as a lib by many Linux systems ... I've learned this from the PHP packager himself for Debian systems ... ("is not and will probably never be provided as a shared object", from memory). FYI, first versions of RayPHP were using system PHP lib :)


I think that I understand why you're working on this to get a cleaner root directory for Raydium, since it's true there's a lot of files out there (1249 files in my main branch, for example), and I'm OK to say it can be way better, even if it's not a real problem for me to have so many files.
In facts, it's not the first time that we talk about this ... but :

- Until ODE is not stable and provided with every distrib with PHP, local libs are the only way to guaranty a simple and consistent building on EVERY Linux system.
- As Raydium developers, we need to have various building scripts : comp, ocomp, odyncomp and ostatcomp are dedicated to help in various tasks : devel, debug, public releasing, ...
- Static building is required, dot. ;)
- IMHO, having only one Raydium build (/usr/lib/libraydium.so.0.0) is an error. A system may host different Raydium games, with different needs. One may ask official 0.680 release, another the 0.700 beta, ... Unix shared object "soname system" is not that good for this (note that libraydium is flagged as 0.0 for now). See libstdc++ chaos, for example :) There's also a problem with external resources (think about rayphp/*).

Apart from this, I think many things can be done:
- You're right, providing a "install" target for the Makefile is not a bad idea. I personally won't use it, but it may allow Raydium installation to be more standard for people that needs it. (What about rayphp scripts ?)
- Split Raydium, apps (games, tests, ...) and data is a must. This thing is on the TODO since ... years :) We must find the best (and simple) way to do it. I love the idea that someone can download Raydium in one time, and can start directly to hack and play with provided tests and games. Perhaps can we split the SVN in two parts but still provide a "all-in-one" tarball ?

Note that my only hope is to keep Raydium SIMPLE to use :)

(I'm writting all this offline, without the help of any web dictionary ... sorry about the probable loads of mistakes ;) )


Top
 Profile  
 
 Post subject:
PostPosted: Tue Apr 11, 2006 11:03 pm 
Offline
User avatar

Joined: Thu Sep 29, 2005 2:59 pm
Posts: 828
i think that raydium is right currently.
Also i'm agree with Xfennec about the las two points: Raydium will have to be splitted in a future and "make install" could be usefull for someone (not me either).


Top
 Profile  
 
 Post subject:
PostPosted: Tue Apr 18, 2006 8:00 pm 
Offline

Joined: Fri Mar 17, 2006 12:41 pm
Posts: 46
Location: Gif/Yvette, France
Thank you for your very interesting reply. I learned a lot about how you see the project.

My point here was "let's have a universal platform" : of course, this would imply :
    :arrow: Backward-compatible API, to let a Raydium-0.68-game work smoothly with raydium-0.7 for instance. Since I'm not a compilation chain expert, I'm not sure it can be done, but I hope. The relevant question is : Is raydium stable enough now to freeze the API (allowing of course new features later)
    :arrow: Full-featured API, with ODE compiled "all the time" : of course, this leads to a slower compilation of libraydium, but not to a slower compilation of the game (except for static compilations, but as you mentionned, it is used only for public releases.

For the "local needs", you made your point, but let's keep this compilation method "troll" aside.

As I mentionned before, my main worry is the structure of the repository : Let me explain how I see it : (raydium/ is of course the SVN root)

Everything non-specific moves to a dedicated Raydium directory :
raydium/raydium as before, but also
raydium/raydium/rayphp and the dependancies
raydium/raydium/php
raydium/raydium/ode

Every game is stored in a dedicated subdirectory :
raydium/skydiver
raydium/kinghill2

raydium/maniadrive
raydium/maniadrive/music
raydium/maniadrive/nusoap
Note : If this technology is to be used elsewhere, maybe raydium/raydium/nusoap is better...
I'll have a look at how you use it and why (I'm finishing network engineering studies, and if I can help, it would be better for the project that I don't touch anything graphical ;) )

and test and demo files are in a tests directory
raydium/tests

The graphics and sounds are downloaded in the game directory, or better, in a "files" subdirectory of the game.

Of course, a lot of paths are to change, but it could be a way to get rid of (possibly) hard-coded values in the code. Instead of changing compilation shellscripts, I think we could use make with several targets for the several compilation methods of a game (just let me dig up my makefile lessons ;) )

I believe this fairly important change in the structure would make us think again about where thing are to be (for example, I believe the database should be in the home directory of the current user) and would allow the development of projects just using the lib "as provided". In my opinion, Raydium is near the point where it can be used by the growing open source games community !

_________________
Phyce


Top
 Profile  
 
 Post subject:
PostPosted: Thu Apr 20, 2006 7:48 pm 
Offline
User avatar

Joined: Sun Mar 16, 2003 2:53 am
Posts: 2591
Location: gnniiiii (Scrat)
Yop,

- Backward-compatible API
I think we can do it. No major API break was done during the whole development, and I think we can continue in that way, thanks the simple "linear" API style (opposing to object oriented API). Another possible option, if we absolutely need to break backward compatibility in the future, is to change project's major version (ex: Raydium 2).

- Full API (ODE + PHP + RegAPI)
Yeah, why not. When I started to work on ODE and PHP, my main idea was to provide theses features as options. I must admit that RayODE and RayPHP became quickly mandatory in my mind :) The only (small) benefit right now is a quicker compilation with small "no-ODE and non-PHP" projects like raydium_modler, so I think we can definitely consider theses API as core ones. (starting with removing *PHP_SUPPORT, *ODE_SUPPORT, ... in config.h)

- NuSOAP
A few months ago, Prospere decided to change the way ManiaDrive was "talking" to website and database, switching from simple HTTP requests to WebServices. As I was wanting to integrate this great change quickly, and since we're always stuck with PHP4 wich is not providing native support for SOAP (unlike PHP5), we decided to temporarily merge NuSOAP to SVN repos. No doubt for me, the good solution here is to switch to PHP5 :)

- raydium.db in user "home" directory
Again, why not. The main problem I see here is that we must support Linux and every windows version. Need some research, for me.

- Makefiles instead of compilation script ?
OK for me, but I want to be able to make a compile+run cycle in one line :)
I like *comp* scripts, since it allows me to open and power-up my laptop anytime, anywhere, logon and enter someting like ./ocomp.sh myapp.c and start working with/on Raydium VERY quickly. Create a new Makefile and do multiples make raydium;make myapp each time I want to test something is not why I call "simple" :)
Perhaps can we provide a better and more powerful version of Makefile, but still offer *comp* scripts. Note that a rewrite of theses scripts is planned since a looooong time to centralize GCC options in one configuration file (or a '...-config' command, see below) instead of duplicate it in every script like it's done for now. Feel free to work on Makefile ! (I think you can contact Mildred if needed, wich wrote the [great] original Makefile)
We must also think about the best way to allow people to link their apps to Raydium using their own building system, perhaps with a 'libraydium-config' binary (like 'libpng-config', for example). This last idea is perhaps the best way to unify Makefile and all *comp* scripts.

- per-app directory in SVN
I think that you can imagine how huge is a such change :) Appart from the "technical" work needed for this new organisation, I'm used to work with "one-big-dir" (c) for years now, and it will be hard to change the way I work :)
BUT, you're right. It's hard to say, but you're right :) The only real problem I see with splitting apps like this, is data redundancy: Raydium applications shares a lot of files each other... That's one of the main ideas behind CQFD data repository: providing a "big" common database of models, textures and sounds for various Raydium apps.
For example:
Code:
[xfennec@Julien opengl]$ du -sh
2,4G    .

... I can't waste more drive space because of duplicated data ;)

IMHO, we must find a way to allow different apps to share data (even if it works only with the default SVN directory structure) and write a new "full" Makefile before starting project tree reorganization.

Don't you think ?


Top
 Profile  
 
 Post subject:
PostPosted: Thu Apr 20, 2006 8:25 pm 
Offline
User avatar

Joined: Thu Sep 29, 2005 2:59 pm
Posts: 828
about per-app directory in SVN
In linux is quite easy to share contents with symlinks. Can this be done in others systems?

I think a good way to make new apps with this method could be:
1-We create the folder with the name of the app
2- we symlinks ALL the common files to that folder
3-the needed files will be overwritten for local versions and the untouched files will be symlinks to the common ones, so a lot of space can be shared.

It's just a fast idea....

BUT

as Xfennec, i think that that change is huge (and requieres a lot of coordination) so should be posposed to the release of maniadriver. So, first we have to "finish" maniadrive and then we can start this change.


Top
 Profile  
 
 Post subject:
PostPosted: Fri Apr 28, 2006 6:06 pm 
Offline

Joined: Fri Mar 17, 2006 12:41 pm
Posts: 46
Location: Gif/Yvette, France
I've thought a bit about a way to share data, portable, without having to rewrite everything. And I come up with something : what about a "media path" that would work like PATH environment variable, ie :
-- search the directories in the path for a given file, and return the first encountered file.
-- if not found, download it from the repository and put it in the default media directory. we could even use a subdirectory by extension.

There would be a command line option to temporary add dirs to the path, and to define default path. And/or a raydium database entry to define these.

This way, existing installations of raydium would continue to work, my dream of a media/ directory could get real, and games could even have a local ./media directory for non-shared media.

_________________
Phyce


Top
 Profile  
 
 Post subject:
PostPosted: Fri Apr 28, 2006 7:56 pm 
Offline
User avatar

Joined: Thu Sep 29, 2005 2:59 pm
Posts: 828
great :)


Top
 Profile  
 
 Post subject:
PostPosted: Fri Apr 28, 2006 9:55 pm 
Offline
User avatar

Joined: Sun Mar 16, 2003 2:53 am
Posts: 2591
Location: gnniiiii (Scrat)
A looong time ago, we've talked about such idea. That was about a packed-data format, like Pak0.pak file in Quake, for example. We didn't did it, since there was an unsolved problem with this idea of providing sort of "fopen()" wrappers: they're only available for Raydium. Libvorbis, PHP and any other linked lib will continue to use regular fopen() function and will not use our smart wrapper :)

Right now, I only see the problem with Vorbis and PHP (Did I miss someting ?), so we must find a way to force theses libs to use OUR fopen wrapper, and this in a portable way. If we can do this, we can do big things :)

As a side note, Sunday I'm on vacation for one week. Away. Far away. Away from my computer (I must resist and not take it with me). ;)

So don't expect anything from me until a week ! You two are now Raydium guardians :)

edit : We should start a new thread, I think.


Top
 Profile  
 
 Post subject: Redifining symbols
PostPosted: Sun Apr 30, 2006 9:05 pm 
Offline

Joined: Sun Apr 30, 2006 8:55 pm
Posts: 22
Location: Plaisir (et oui, ça fait plaisir)
Here is a way to force to use some symbol of one library before other.

First, some code :

Code:
#include <stdio.h>
#include <stdlib.h>
#include <dlfcn.h>
 
FILE * fopen(const char * path, const char * mode)
{
        FILE * (*orig_fopen)(const char *, const char *);
        void * handle;
        char * error;
 
        handle = dlopen("/lib/libc.so.6", RTLD_LAZY);
 
        if(!handle)
        {
                fputs(dlerror(), stderr);
                exit(EXIT_FAILURE);
        }
 
        orig_fopen = dlsym(handle, "fopen");
 
        if((error = dlerror()) != NULL)
        {
                fprintf(stderr, "%s\n", error);
                exit(EXIT_FAILURE);
        }
 
        fprintf(stdout, "Opening %s (%s)\n", path, mode);
 
        return orig_fopen(path, mode);
}


Then, we make a library.

Code:
gcc -fPIC -rdynamic -g -c -Wall lib.c
gcc -shared -Wl,-soname,lib.so -o lib.so lib.o -lc -ldl


Finally,

Code:
export LD_PRELOAD=~guitou/lib.so
./test


That is not what you can call a portable way but it works :)


--
guitou


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 01, 2006 8:15 am 
Offline
User avatar

Joined: Thu Sep 29, 2005 2:59 pm
Posts: 828
Maybe i'm a bit confused but...
The problems come as you want to do fopen ("example.tri") from a regular c file, or inside php or ode libs,right?
Why not then use a second function inside fopen?
Something like filename("example.tri")
So the final call will be fopen(filename("example.tri"))

filename is a function that searchs in the know "raydium-path" for the file and returns the first one that exists. And in this way there is no problems... a part from the need to recode all the calls to a file.:)


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 01, 2006 12:57 pm 
Offline

Joined: Fri Mar 17, 2006 12:41 pm
Posts: 46
Location: Gif/Yvette, France
the last revision (301) introduces changes about that : I believe (hope?) all the files are now opened via raydium_file_fopen().

For the moment, the mediapath is hardcoded to ".:media" and I've been able to use the following script :
Code:
#!/bin/sh
mkdir -p media
mv *.tga media
mv *.tri media
mv *.wav media
mv *.ogg media
mv *.mni media
mv *.prt media

without problems.

Following the goal of tidying the root, we have to think how and where files are going to move :
-- how will we manage game-specific data ? a media/ subdir in each game, keeping the global media/ directory ? what about music tracks ? Where are we going to put R3S downloaded files ?
-- how will we compile (makefile and/or scripts) games, ensuring raydium is up to date, with a clear packaging allowing games to be distributed separately ? (I hope all this will end up in distributions one day)

By the way, it's a bit awkward to introduce him this way, but Guitou is a friend of mine, I asked some advice and he started looking at the code.

_________________
Phyce


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 01, 2006 6:25 pm 
Offline
User avatar

Joined: Thu Sep 29, 2005 2:59 pm
Posts: 828
aham.
I'm not sure it'sthe better thing making a huge change like this without the "father of the child" around here :)
Anyway, if it's working, then it's ok

Maybe it's a good moment to comment a new feature i liked to enter. An update system to R3S repo.
So, when an binary is launched, optionally, the repo can be checked to new versions of the media files.


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 01, 2006 7:17 pm 
Offline

Joined: Fri Mar 17, 2006 12:41 pm
Posts: 46
Location: Gif/Yvette, France
Nice idea !

BTW, my modifications weren't so huge (just the path search thing and reading WAV files into a buffer instead of opening it with OpenAL), but it makes me confident about the whole "folderisation" thing.

There is a lot of features that can be added in this raydium_file_fopen() function :
-- the one you describe above,
-- a non-hardcoded path, using database, or command line, or both
-- the ability to download files from R3S in a defined directory (not the root one) (to be discussed, how do we know were a file is to be put ?)
-- and much more !

_________________
Phyce


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 01, 2006 8:09 pm 
Offline

Joined: Sun Apr 30, 2006 8:55 pm
Posts: 22
Location: Plaisir (et oui, ça fait plaisir)
To vicente:

I totaly agree with you. All function opening files in code managed by raydium should use a custom function as you described or as phyce has done in the last revision.

The original problem was, from my point of view, with external libraries such as openal which open files with fopen. The solution I proposed was only here to show that it is possible to map all fopen calls to a function we wrote. I don't like it much either for it is a dirty hack and it can have way too many side effects :)

Oh and yes, I didn't introduce myself properly. Thanks to phyce who has done it ;)
He spoke about raydium, I gave it a try and I find it quite interesting :)


--
guitou


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 31 posts ]  Go to page 1, 2, 3  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 29 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group