gmane.games.devel.sweng

Re: Hello

metanet software — 2013-03-30T23:35:58

http://www.simamex.com/gpi/mgs.kitb?zzw    

      
 

    

metanet software





























































  3/31/2013 12:35:58 AM
 






















xj
                                                         
3/31/2013 12:35:58 AM 
metanet software_______________________________________________
Sweng-Gamedev mailing list
Sweng-Gamedev< at >lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com

Re: Sweng-Gamedev Digest, Vol 71, Issue 18

Gabriel Sassone — 2013-03-30T00:11:39

Thanks for all the reply guys, much appreciated.
The links you wrote are a lot helpful to have a deeper insight about this
challenge!

Still the solution to this problem really depends on the scope of the
project you're working on.
So far the main separation is between the reflection API and the way to
feed it, in one of the 5 methods.

I will continue experimenting with it, even though already having setting
up a simple api with fields and types showed me the power of this approach.
Sure there are limitations, and with complex memory layout you can bump in
some problems (virtual or multiple inheritance, even though are not my
case), but gives you a solid framework to start looking at things done with
less code, less errors and concentrating more on algorithms more that data
conversion!

I was thinking also about pointers to object and memory allocation: when
you are serializing/deserializing a pointer to an object, what is your
strategy?
You want to have a building order of objects, taking track also of the

Re: On the benefits of reflection

Massimo Del Zotto — 2013-03-20T06:48:45

My question regards the goal I'm targeting. When something is worth, I can
perceive a strong set of goals "opening" at a lower effort. I don't have
that feel with reflection. I am not really able to pull out the exact
question synthethizing my doubts but I hope I will be able to explain
myself with various comments.

So from reflection we're obviously able to enumerate the fields of any
type, get the type of field, its attributes, and from reflection and
invoking, we can modify the field and notify the class of changes from
editor-side only if needed.  Those attributes and notification methods can
easily be compiled away in final builds if need be.

The texture example does not fit my likings at all, perhaps I'm stuck in a
old-school reflection line of thinking but it looks to me it would be much
nicer using a component based system. Do component based systems qualify as
reflective systems?

Reflection can tell us what needs to save to disk or what needs to be
serialized over the network.  Sure you can write

Re: On the benefits of reflection

Tinco Andringa — 2013-03-19T21:56:28

So it kind of works sort of like an inverted events system. For
example using events:

class Object {
  TakeDamage(int damage) {
      var total = max(armor - damage, 0);
      hp -= total;
      Events.Trigger(new DamageTakenEvent(this, damage, total));
  }
}

Events.On<DamageTakenEvent>( e =>
  Console.WriteLine(e.Object.Name + " took: " + e.total + " damage.");
);

Using aspect oriented programming:

class Object {
  TakeDamage(int damage) {
      var total = max(armor - damage, 0);
      hp -= total;
  }
}

AOP.Around(Object.TakeDamage, (invocation, params, object) =>
   var total = object.HP;
   invocation.proceed(params);
   total -= object.HP;
   Console.WriteLine(object.Name + " took: " + total + " damage.");
);

Note that I don't really know what C# aop looks like, I'm not sure you
can be that succinct when using method references (delegations) but
that's the gist of it. You can see now the game logic function
TakeDamage has no awareness of any observers, so there's a clean
separation of concerns.

Re: On the benefits of reflection

Douglas Cox — 2013-03-19T21:17:05

That was not exactly C# despite what it looks like -- it's our own variant
and compiler.  The GC uses minimal reflection in the sense that our type
information includes a small bit of information (provided by the compiler)
useful for tracing through object references.  It's a multi-generational
GC, a bit tailored to our needs.  It supports finalizers, deals with
multiple threads, etc.  It is used pretty much like the one in C#.

Honestly, after 12 years we just got tired of C++'s issues (see this thread
for a good example), and so far love how clean it is to write the new code.
 When we need to write some C/C++ code it's as easy as:

void Foo()
{>
   // c++ code goes here.
<}

It may sound like a pain to get working, but if you look at some of
runtime-recompile work, and then realize that most of the annoying parts of
that can also be hidden behind a 'pre-compiler', it isn't as bad as it
sounds.  Especially when you know the data layout of your own vtables and
can run through all allocations fixing up point

Re: Reflection in game engine (c++)

Douglas Cox — 2013-03-19T20:40:14

"That system continues to work with dlc and patches, but if your patch
needs to change a type, you'd probably be sending new data files using that
type anyway."

You'd think so, but we've had a few examples of this already in the last
few years.  Only a few new instances needed to override a default value.

"We usually read these files on a separate thread, and it's comforting not
to be making lots of allocs that might fragment memory if interleaved with
allocations that might happen on other threads."

This seems like it would happen regardless unless your main 'runtime'
allocations are coming from a different memory pool.

I probably should have mentioned that I was also talking about overriding
values on 'prefabs' with this same system.  So when we override just a few
position/rotations it is saving considerably more than flattening the
entire prefab into the level file for instance.

Sorry I should have made this a separate thread.

-Doug


On Tue, Mar 19, 2013 at 4:02 PM, Adrian Stephens <adrians< at >isopod

Re: On the benefits of reflection

Tinco Andringa — 2013-03-19T20:38:21

You use reflection to implement your own garbage collector? This is C#
right? Does your GC just collect resources that aren't needed anymore
or does it actually do a better job of managing memory etc?

To add to the discussion about benefits of reflection:

Coming from Ruby I'm always looking for ways to make things easier to
express using reflection. I think serialization/deserialization is
just one usecase for this. For example C# is a language that's more
geared towards object oriented architecture than to entity systems,
but using reflection you can generate the code that links your
components together at runtime so there's not a lot of boilerplate
code required for every component.

Sort of related to reflection, recently I made a game as an academic
exercise in javascript using an aspect oriented programming library.
AOP has a similar philosophy of taking separation of concerns to the
next level. It worked out very well, you can have the game engine
drive the views without having any hooks or view rel

Re: On the benefits of reflection

Douglas Cox — 2013-03-19T20:21:10

I'm not entirely sure what your question is (perhaps an example of what
you're trying to ask may help), but I can tell you what we would do and
what the benefits are. This example is not real code, but it shows

class Texture
{
    [EditableFilename( "png" )]
    String filename;

    [Editable( onChanged=OnFormatChanged )]
    TextureFormat format;

    [EditableFloat( 0.0, 1.0 )]
    float scale;

    TextureData data;

    void OnFormatChanged()
    {
       ... re-cache texture on PC ...
    }
}

// Just some example of how we access the type info.
void DoSomethingWithFields( Object object )
{
    foreach( var field in typeof( object ).fields )
        Console.WriteLine( "Field: {0} {1} = {2}", field.type.name,
field.name, field.GetValue( object ) );
}

So from reflection we're obviously able to enumerate the fields of any
type, get the type of field, its attributes, and from reflection and
invoking, we can modify the field and notify the class of changes from
editor-side only if needed.  Those attribute

Re: On the benefits of reflection

Marc Hernandez — 2013-03-19T20:09:37


  I havent seen a non reflection based network system that was as easy as a
reflection based one.  By serialization, if you mean:
send()
  health >> out;
  pos >> out;

recv()
  health << in;
  pos << in;

  A reflection system merely automates this and makes it very difficult to
get wrong even with rapid development.  Its a completely DRY (dont repeat
yourself (haha)) architecture.  With your traditional serialization you
have to touch 3 places in code when you change some structure, whereas with
reflection I only change one.  If you want to make it simpler it just
starts getting into adhoc reflection systems.



  Sorry, what I meant was my config system uses many of the same classes as
the networking layer.  The only real difference is config uses my XML
read/write and goes to and from disk while my network does binary
read/write and goes to stream.  I also save off the binary network stream
for trivial debugging later via read(binary) -> write( xml ).  All from
fairly short classes I wrote once.



  I

Re: Reflection in game engine (c++)

Adrian Stephens — 2013-03-19T20:02:43

We flatten our data, so reading data in-game is just a question of allocating enough space (from the different flavors of memory - normal, video, sound, etc) reading the data and patching the results.  During development there is the possibility of reading some out-of-date types, and those need to be converted  during the patch-up, which means they do take extra memory.  The shipped code shouldn't have to do that.

That system continues to work with dlc and patches, but if your patch needs to change a type, you'd probably be sending new data files using that type anyway.

We usually read these files on a separate thread, and it's comforting not to be making lots of allocs that might fragment memory if interleaved with allocations that might happen on other threads.

Regarding storage space - if you compress your binary files, as we do, any often-used default values are likely to compress away very easily; and since our data is flattened and compressed during development, there's no additional code to worry

Re: Reflection in game engine (c++)

Jarkko Lempiainen — 2013-03-19T19:58:39

While ago I wrote an introductory article to C++ reflection system I
implemented, which may answer some of your questions about reflection in
general as well:
http://community.spinxengine.com/content/40-introduction--sxp-reflection-sys
tem.html

Simply put a class reflection exposes class members in run-time and you can
build other more complex systems (e.g. networking, editor property grid,
versioned save/load, etc.) on top of it without having to write specific
code for each class for these systems.

 

 

Cheers, Jarkko

 

 

From: sweng-gamedev-bounces< at >lists.midnightryder.com
[mailto:sweng-gamedev-bounces< at >lists.midnightryder.com] On Behalf Of Massimo
Del Zotto
Sent: Tuesday, March 19, 2013 3:23 PM
To: sweng-gamedev< at >midnightryder.com
Subject: Re: [Sweng-Gamedev] Reflection in game engine (c++)

 

As a side note, the Bullet physics API serializes everything with its
accompanying DNA as far as I know.

I'd just throw the link in for documentation.

http://bulletphysics.org/mediawiki-1.5.8/index.php/Serial

Re: Reflection in game engine (c++)

Adrian Stephens — 2013-03-19T19:43:48

Yes, sorry - the first part of my response was addressing keeping C header files with the assets.  When I re-read what you wrote I realized you were talking about keeping the metadata as part of each asset, and I agree with that.  My second point was that external tools need to allow people to create new entities based on some list of templates, and for that the tools still need access to those definitions.  If all you need to do is 'process' existing assets, the embedded data would be enough. 

If I add a new variable to my textures, old textures will still load and be given the default value for that field.  I think we agree on why and how that's a good thing!

I do use the same reflection system for all the data - there really are no special cases - but the the language you use to specify a type definition is always going to be turned into some internal representation at some point; how you express those types is not as important as the ability to treat the data in a uniform way in code.  So -  in the in

Re: Reflection in game engine (c++)

Massimo Del Zotto — 2013-03-19T19:23:18

As a side note, the Bullet physics API serializes everything with its
accompanying DNA as far as I know.
I'd just throw the link in for documentation.
http://bulletphysics.org/mediawiki-1.5.8/index.php/Serialization_DNA_restrictions,_size_and_alignment_rules

I have no idea if this qualifies as "reflection" but it seems Jarkko'
suggestion is really good habit to me.

Massimo
_______________________________________________
Sweng-Gamedev mailing list
Sweng-Gamedev< at >lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com

Re: On the benefits of reflection

Massimo Del Zotto — 2013-03-19T19:15:06

Let me elaborate on those points so I hope people can get in my point of
view.

1- "Trivial" network system. Done for ages, before reflection was in vogue.
Considering how UDK manages that thing, I believe we're cutting a lot of
corners. What's the problem in just doing serialization in the standard way?
2- Config system. No idea what "the same hardware" is supposed to mean. I'd
be careful in making my networking subsystem work the same as a disk IO.
3- save load. In my system, just restoring VM state is going nowhere. Maybe
the scripts will just churn along. Sure engine state isn't going to make it
there.
4- Message passing... through the networking system? I don't see the
connection with reflection. I'd be extra super careful in doing that.
Message passing to who? For what purpose? At what frequency?
5- I am using reflection myself in my system as well. Good for your friend.
Perhaps you could contact him and ask what exactly made his work easier?
I'd be interested in considering those cases. As I said, I h

Re: Reflection in game engine (c++)

Jarkko Lempiainen — 2013-03-19T18:26:04

If you plan to interpret data, you need to parse those class definitions
anyway, regardless how you define & store them. Furthermore, you *have to*
embed the definitions in asset files if you don't want to do mass asset
conversions every time you change definitions + you can't guarantee access
to all the data using specific definition anyway. These asset files also
need to store only definitions for objects that are stored within that given
asset file, not all your definitions (wasn't sure if this was what you
actually meant, but wanted to clear that up).

 

So, if you add a new variable to your textures (e.g. color space variable to
define how to interpret the texture), you have to rebuild all your texture
assets? I don't see why you wouldn't just use the same reflection system for
all the data. Another example is container classes, which I have seen
requiring special treatment by reflection systems. I think reflection system
needs to be powerful enough to be usable for all type of data and that you
should

Re: On the benefits of reflection

Marc Hernandez — 2013-03-19T18:09:47

Man, I love reflection.  It allows me to just belt out working systems with
ease.

So, Im using it for:
x) Trivial network system.  All my network packets are just classes that I
marked [Serializable].  If I have any special handling I use custom field
attributes to mark it up.
x) Config system.  Uses all the same hardware as the networking system, now
its just loading to and from disk.
x) Save/Load.  Same.
x) Message passing system.  This is essentially just the networking system.
y) I have a friend that built a nice level editor in XNA in a month which
heavily relied on reflection.

  With all of these I have 1 reflection handling class.  Currently its a
custom written XML reader/writer that slightly abuses the XML rules so
files are small.  I also have a binary reader and writer.  The networking
system can negotiate for which one to use.  Making a viewer looks like:
structure = Read( BinaryFormatter );
Print( XMLFormatter );

  I then use reflection to automatically type safe handle the messages.
 So, you

On the benefits of reflection

Massimo Del Zotto — 2013-03-19T17:35:06

Hello,
this question spawned from the recent discussion on reflection. I guessed
it was better to open an independent discussion instead of hijacking the
thread.

2013/3/19 Douglas Cox <ziflin< at >gmail.com>


I'd be interested in knowing more about doing the other thing around.
I have a small little language I've been using for a while and I've been
fairly happy with it. So far, the language itself does not have reflective
constructs (nor I plan to add them) but I use reflection internally mainly
to simulate duck typing.
So far, I haven't well understood the benefits of reflection. I had seen
some examples in ObjC but they didn't really appeared to be buying much
from me. I know reflection is there in a Java package, but I couldn't quite
figure out the whole picture.

Could you please elaborate on the benefits of using reflection? Please be
specific on whatever you're talking about reflection from the native core
or reflection from the language.

Thank you
Massimo
_______________________________________________

Re: Reflection in game engine (c++)

Douglas Cox — 2013-03-19T16:09:35

One issue that we were thinking about after coming up with a file format
using reflection that allows for data-derivation, versioning, etc., and
that basically looks like "field name + field type + value" (only
overriding/saving values that were changed) was: "Is it worth trying to
optimize or flatten this for final builds?"

At first we were just going to write a different serializer and write out a
flattened set of in-order values thinking that our types would never
change.  But with DLC and patches, this is not the case.  We may release a
patch that changes a type (adds or removes a field) and existing
(non-patched) data files would fail to load.

Another reason to not flatten is that it is quite possible that the final
data in the 'overrides only' format would be much less storage than the
flattened format.  And of course not having to write additional code that's
only run in final builds is a plus.

Anyone else have thoughts on this or do it differently?

--
And as far as the debate over where to put th

Re: Reflection in game engine (c++)

Mike Shaver — 2013-03-19T04:07:58

I wanted to just have one place that things were declared, and for that to be with real code rather in a separate "please don't edit this no matter how much VC++ tells you this is the definition" file.

My point is really that it's over-constraining the problem to choose one format that is both fast+compact enough for runtime use, and flexible+straightforward enough to bend into all the parts of the tool pipeline.

It's the work of 5 lines of Python to go from JSON to XML or CSV or .ini files or whatever. That someone has linked the format parser into a dozen different software environments is indeed an advantage that I would take seriously. I generally don't know what my tools are going to be until I've written them, so free option value is pretty nice.

I've done the "IDL to headers and packed metadata files" thing, and it was pretty annoying (and error-prone) to wire it into even a half-dozen straight-C tools. I may be overreacting to that experience, but mutatis mutandis I very much prefer one tiny expor

Re: Reflection in game engine (c++)

Adrian Stephens — 2013-03-19T02:58:48

So you're writing macros to turn your C structures into data, building them into a library, linking it with your exporter and running the exporter to generate a script-format of the structures.  Which, then has to be parsed by your tools.  I honestly didn't mean that to sound so negative, but wouldn't it have been easier the other way round - start with a simple domain-specific language to define your types and spit out C header files?

The PHP thing works because you're proposing outputting JSON, which somebody presumably has already linked into it.  If that sort of thing is difficult you could certainly output JSON at the same time you output C header files, or I suppose you could probably even use JSON *as* your script format.


On Mar 18, 2013, at 5:26 PM, Mike Shaver wrote:


_______________________________________________
Sweng-Gamedev mailing list
Sweng-Gamedev< at >lists.midnightryder.com
http://lists.midnightryder.com/listinfo.cgi/sweng-gamedev-midnightryder.com

Re: Reflection in game engine (c++)

Adrian Stephens — 2013-03-19T02:44:21

Yes, you *can* keep your class definitions with the assets, and have external tools read them - but my point is that if you're going to be writing something that parses these definitions, you don't gain much by defining them using contrived macros and templates.  I am proposing defining them in a simpler, clearer way that can be easily parsed and spat out as headers, or whatever else you need.

Assets can embed their structure definitions within themselves - they're trivially small binary packets - and since tools contain their own sense of how a 3d scene, say should be stored, the metadata system can modify the data on the fly, rather than require everything to be re-exported because you added a 'flags' field (a rule of game development is that *everything* needs a flags field sooner or later!)

Actually, I just re-read your first paragraph and I think I misunderstood what you were saying.  So I'll amend my response to say that unless you store *all* your definitions in *all* your assets, you still won't be