Draft: Migrate object hierarchy to ECS

langurmonkey commented

2022-05-02 14:19:54 +00:00

(Migrated from gitlab.com)

This merge request contains the work needed to migrate the internal model to an ECS.

Gaia Sky now uses a scene graph which is baked into an OOP inheritance hierarchy that has become a major PITA. It is huge, with multiple overrides and a far too deep structure. It is difficult to maintain and prone to errors.

The idea is to move the whole object hierarchy (the root being SceneGraphNode) to an entity component system (ECS) paradigm. Artemis-odb and Ashley were considered. Artemis seems a bit over-engineered, so we'll go with Ashley for now. Additionally, Dominion is looking very compact, fast and simple, but it is still in a very early development phase. It may be worth a look in the near future.

The work will be split into phases:

The first phase moves the JSON loading and recreates the whole structure into entities and components.
The second phase moves the logic in the scene graph objects into systems and tries to achieve a fully working build.
The third phase also incorportaes the current renderer into the ECS structure.

Once this is implemented, the code base will be wildly different and much more maintainable. It will enable easier parallelization in two different ways:
First, some of the update systems will be able to run in parallel. When the first top-down update of the scene graph has been completed, the rest of the update operations can essentially be run in parallel. This is not possible right now, as the update operation is monolithic and sequential for all objects.
Second, it will possibly enable concurrent update/render processes (i.e. run the update thread separate from the main render thread). Right now, the update/render cycle is ran in the same thread:

stateDiagram-v2
    [*] --> update1
    update1 --> render1
    render1 --> update2
    update2 --> render2
    render2 --> update3
    update3 --> ...

After this is implemented, the render 1 and update 2 tasks will be concurrent:

stateDiagram-v2
    [*] --> update1
    update1 --> render1
    update1 --> update2
    update2 --> render2
    update2 --> update3
    update3 --> ...

Since the model is being rendered and updated at the same time, this may require an extra 'extract' step, where a representation of the scene containing the information to render is extracted to send into the render system. Once this extraction operation is done, and since the render system reads the extracted data only, the next update cycle can start safely.

All in all, this is a major refactoring and it is expected that it will require a few months' work. The work has already started, and lives in the branch ecs.

Usage

The refactor lives in the ecs branch. Check it out and run it normally:

git checkout ecs
gradlew core:run

Currently, both the old and the new model and renderers live alongside each other, so the frame rates are a bit lower than usual. The default mode uses the old renderers. Use the equals key = to switch between the old and the new, ECS-based model and renderers. EDIT 2022-09-22: The old model and renderers have been removed, so the = is now unmapped, and it is no longer possible to use them.

Progress

Load basedata package into ECS (100%)
Move all loading to ECS (100% )
Implement scene graph logic into systems (100%)
- Initialize (100%)
- Update (100%)
- Extract (100%)
Move render process to systems (100%)
Migrate camera focus to ECS (100%)
Make sure tessellation works
Game mode / elevation maps
SSR
Shadow maps
Scripting API (100%)
Make sure keyframed camera paths system works (100%)
Post-processing (100%)
- Ray-marching shaders and objects (100%)
Spacecraft mode (100%)
Dataset descriptors, filters, attributes, etc. (100%)
Make sure all test and showcase scripts work (100%)
Remove old inheritance model (100%)
Test thoroughly, fix all bugs (100%)

This merge request contains the work needed to migrate the internal model to an ECS. Gaia Sky now uses a scene graph which is baked into an OOP inheritance hierarchy that has become a major PITA. It is huge, with multiple overrides and a far too deep structure. It is difficult to maintain and prone to errors. The idea is to move the whole object hierarchy (the root being `SceneGraphNode`) to an entity component system (ECS) paradigm. [Artemis-odb](https://github.com/junkdog/artemis-odb) and [Ashley](https://github.com/libgdx/ashley) were considered. Artemis seems a bit over-engineered, so we'll go with Ashley for now. Additionally, [Dominion](https://github.com/dominion-dev/dominion-ecs-java) is looking very compact, fast and simple, but it is still in a very early development phase. It may be worth a look in the near future. The work will be split into phases: 1. The first phase moves the JSON loading and recreates the whole structure into entities and components. 2. The second phase moves the logic in the scene graph objects into systems and tries to achieve a fully working build. 3. The third phase also incorportaes the current renderer into the ECS structure. Once this is implemented, the code base will be wildly different and much more maintainable. It will enable easier parallelization in two different ways: First, some of the update systems will be able to run in parallel. When the first top-down update of the scene graph has been completed, the rest of the update operations can essentially be run in parallel. This is not possible right now, as the update operation is monolithic and sequential for all objects. Second, it will possibly enable concurrent update/render processes (i.e. run the update thread separate from the main render thread). Right now, the update/render cycle is ran in the same thread: ```mermaid stateDiagram-v2 [*] --> update1 update1 --> render1 render1 --> update2 update2 --> render2 render2 --> update3 update3 --> ... ``` After this is implemented, the `render 1` and `update 2` tasks will be concurrent: ```mermaid stateDiagram-v2 [*] --> update1 update1 --> render1 update1 --> update2 update2 --> render2 update2 --> update3 update3 --> ... ``` Since the model is being rendered and updated at the same time, this may require an extra 'extract' step, where a representation of the scene containing the information to render is extracted to send into the render system. Once this extraction operation is done, and since the render system reads the extracted data only, the next update cycle can start safely. All in all, this is a major refactoring and it is expected that it will require a few months' work. The work has already started, and lives in the branch [`ecs`](../../tree/ecs). ## Usage The refactor lives in the `ecs` branch. Check it out and run it normally: ```bash git checkout ecs gradlew core:run ``` Currently, both the old and the new model and renderers live alongside each other, so the frame rates are a bit lower than usual. The default mode uses the old renderers. Use the equals key <kbd>=</kbd> to switch between the old and the new, ECS-based model and renderers. **EDIT 2022-09-22:** The old model and renderers have been removed, so the <kbd>=</kbd> is now unmapped, and it is no longer possible to use them. ## Progress - [x] Load basedata package into ECS (100%) - [x] Move all loading to ECS (100% ) - [x] Implement scene graph logic into systems (100%) - [x] Initialize (100%) - [x] Update (100%) - [x] Extract (100%) - [x] Move render process to systems (100%) - [x] Migrate camera focus to ECS (100%) - [x] Make sure tessellation works - [x] Game mode / elevation maps - [x] SSR - [x] Shadow maps - [x] Scripting API (100%) - [x] Make sure keyframed camera paths system works (100%) - [x] Post-processing (100%) - [x] Ray-marching shaders and objects (100%) - [x] Spacecraft mode (100%) - [x] Dataset descriptors, filters, attributes, etc. (100%) - [x] Make sure all test and showcase scripts work (100%) - [x] Remove old inheritance model (100%) - [x] Test thoroughly, fix all bugs (100%)

langurmonkey commented

2022-05-02 14:19:55 +00:00

(Migrated from gitlab.com)

requested review from @langurmonkey

langurmonkey commented

2022-05-02 14:19:55 +00:00

(Migrated from gitlab.com)

assigned to @langurmonkey

langurmonkey commented

2022-05-03 08:51:52 +00:00

(Migrated from gitlab.com)

added 1 commit

a0d6ece9 - none: add set-up stage to initializers. Orbits initialize (partially).

Draft: Migrate object hierarchy to ECS #655

Usage

Progress