Sabbatical Log: November 30th

This blog series is running about a week behind my actual work, giving me time to clean things up.  The date the actual work was done is in the title, horizontal lines indicate when I stopped writing and started coding.

It’s my last official day of sabbatical! I’m still planning to do a retrospective post, and have the weekend to maybe do some extra-sabbatical work.

But today, I’m hoping to finish up the sword knight.

I’ve got some smarter state management on the knight, it will no longer wander too far away from its origin, and have made the sword sharp. Now, when it’s in motion, it will “do damage” to anything it comes into contact with that can receive damage.

Right now bushes are the only things take take damage, so the knight is limited to cutting them. This is, again, emergent behavior – neither the knight’s code nor the bush’s code knows about the other and the collision system is completely ignorant of damage. I continue to be impressed with the Entity-Component-System design pattern, which made this structuring pretty painless.

Now to get the knight to see (and subsequently chase) the player.

Well, I didn’t quite get to chasing… but the knight can see things now. I modeled is vision as an invisible, but still hitable, box – so it’s just a matter of adding a reaction to the player colliding with it.

Turning on one of the debug layers lets us see the vision cones in action.

Continue onto December 1st and 2nd

Sabbatical Log: November 29th

This blog series is running about a week behind my actual work, giving me time to clean things up.  The date the actual work was done is in the title, horizontal lines indicate when I stopped writing and started coding.

Work continues on the sword knight. There’s a lot of infrastructure that has to be setup, including lots and lots of assets and animations, which is time consuming.

Currently, all I’ve got is a dismembered knight who can look right.

I should have something more impressive later today.

Progress continues, animations are as always pretty time consuming.

Thought this buggy loop looked funny though.

I’ve now got all the basic logic for the sword knight going.

It basically has three states:

• Standing
• Walking
• Searching

You can see the various states here

Obviously, I haven’t actually got the knight moving yet – that shouldn’t be too tricky. But first I’ve got to define all the hitmaps for the various pieces (the knight is modeled as five entities: feet, body, head, sword, and shield), or bad things will happen when the knight starts walking into things – and that’s going to take a bit of time.

After that I need to make the knight not blind, right now it can’t actually find anything when searching. Fixing that will require a temporary entity, and a new state where the knight chases the player. And then finally, I can implement behavior for when the knight’s sword hits a player and the player sword hit’s a knight (or the knight’s shield).

With my time running out, I don’t know if I’ll be able to do all of these but I’m going to make an honest attempt.

End of the day, and the sword knight can move and has hitmaps for all his components.

It’s collision logic is mostly working, though there are no programmed behaviors for such collisions yet. There’s also no bounds checking to make sure the knight doesn’t just wander off the map.

The knight is thoroughly exercising all the patterns I’ve built up, and I can see plenty of places I’d make improvements – but there’s no time for it. One thing I did do was create an AssociatedEntityComponent that makes it easy to group entities together, an idea I’ve been putting off (it does complicate the EntityManager class) but keeping track of the 5 different entities making up a sword knight made it necessary.

Currently the SwordKnightSystem is 861 lines of code, it’s chunk of the ObjectCreator is 612, and the SwordKnightStateComponent is 74. Lots of code is spent dealing with error cases, and there’s lots of room to improve the ergonomics of the code… but all told, I’m not too displeased with current state of things.

I’m going to return to the sword knight tomorrow, hopefully wrapping it up.

Continue onto November 30th

Sabbatical Log: November 28th

This blog series is running about a week behind my actual work, giving me time to clean things up.  The date the actual work was done is in the title, horizontal lines indicate when I stopped writing and started coding.

I’ve got a little bit of cleanup to knock out before getting to implementing an enemy, and I want to get allocated bytes and some GC states into a debug overlay. So I’m going to do that real quick.

Humorously, I did find a couple leaks with this new GC reporting – and most of them were in the “render this debug layer”-code.

(generations go left to right, so it’s 5 gen0 collections, 4 gen1 collections, etc.)

I did also find a bug with returning FrameState’s to a reused pool, and fixed it. There’s a more complicated issue around delayed loading of some types & delegates, in particular those around collision detection, which can cause some collections after reaching the stable state.

I could go and fix those, but they’re a fixed cost and frankly I want to get started on an enemy.

I’ve decided I’m going to implement a sword knight, my first enemy and probably my last task for this sabbatical.

Continue onto November 29th

Sabbatical Log: November 27th

This blog series is running about a week behind my actual work, giving me time to clean things up.  The date the actual work was done is in the title, horizontal lines indicate when I stopped writing and started coding.

I’ve got falling down a pit mostly working, but I want to add a little drop shadow to show where the player is falling. This will be the first exit transition that creates an entity, so that’ll be interesting.

Whelp, that’s wrapped up

Again, I’m not gonna win any animation awards – but it works.

I think my final task for this sabbatical month is going to be to implement an actual enemy to fight, but before I do that there’s one more piece of infrastructure I want to build: a “job system”.

Pretty much everything that matters has multiple cores now, and it’s kind of silly not to take advantage of them – but multi-threading has some complications in the context of a game. Whereas a web app can assign threads to different logical requests, games have a single logical process. Unlike web apps where all threads are more or less equal, games typically have a privileged thread (the UI thread). And since this whole endeavor is a learning exercise, I think I should play around within these constraints.

I’ve worked up a very simple job system, composed of 3 parts:

• JobRunner
• Job (which implements IJob)
• JobsCompletionToken

JobRunner is pre-sized for a set of threads, and a maximum number of concurrent jobs. Other code will use a single JobRunner to pre-allocate Job’s based on a delegate, and whatever additional state they need (the current GameState will always be passed, as it should always be needed). JobsCompletionToken is returned by the JobRunner.StartJobs method, wraps around whatever IJobs are passed, and has a WaitForCompletionMethod().

The basic idea is that JobRunner pre-allocates threads and JobCompletionTokens, various bits of code create their Job’s when they’re spun up, those Job’s (as IJobs) are re-used to run the code but with an updated GameState. I built it using the standard Thread, Monitor, and Interlocked classes. Threads do the actual work (the main thread is never stolen), the Interlocked class is used to add and remove IJobs and JobsCompletionTokens from queues, and Monitor is used to pause and resume threads. Since everything is pre-allocated, once startup is done this runs with no allocations.

Multithreaded code is difficult, and I’m not convinced I got this correct (though I did write tests). It’s also difficult to describe, so I’m just going to link to it: JobRunner, Job, and JobsCompletionToken.

Now to actually use it.

It’s the end of the day and I’ve converted two systems to use the JobRunner: the CollisionDetectionSystem, and the UpdatePositionSystem. Both of these were doing four passes of the same logical step, one for each level of a room.

(single threaded is on the left, multi-threaded on the right)

If you squint you can see some small improvements in DEBUG, although they’re mostly washed out in RELEASE builds. The multi-threading for the collision system is probably too coarse, honestly, since most everything is on one level. Regardless, I feel this exercise was a good one.

Tomorrow I’ll start on a proper enemy.

Continue onto November 28th

Sabbatical Log: November 26th

This blog series is running about a week behind my actual work, giving me time to clean things up.  The date the actual work was done is in the title, horizontal lines indicate when I stopped writing and started coding.

It’s the start of the final week of my sabbatical. I’m still deep in the “move components to an object pool”-branch this morning, but I expect to finish up soon. While I’m in here, I’m also reducing some repetitive passing of new Ids and trying to make the ObjectCreator class (which is responsible for creating room objects at room load time, swords when the player starts swinging, etc.) exhaustively testable.

I’ve finished getting components moved into object pools. As I said earlier, I also added some tests that verify that nothing gets leaked if an error occurs during “object”-creation – where anything is a Component, or an Entity.

When I first implemented those tests, I found I was actually leaking all over the place in failure cases. The causes were a mix of not accounting for errors at all, and forgetting to free components that I failed to attach to an entity.

To make those tests, rather than mock the entire EntityManager I added some bookkeeping that tracked how many “fallible calls” happened and added a switch to force failure after so many of those calls. Then it was just a matter of determining how many calls each sort of object needed, and then failing at N calls, N-1 calls, N-2 calls, and so on. A few objects (like the player) were created via different code paths, so I moved those to the ObjectCreator class and added tests specifically for them (since they aren’t defined via room templates).

I’ve seen it before, but this whole exercise really drove home how much boilerplate code you end up with if you’re manually managing resources. Here’s the code for creating a flower, I’ve marked the boilerplate that handles failures:

As you can see, it’s north of 50% of the code. The whole ObjectCreator class went from 330 lines when I wasn’t doing things correctly, to 1,498 lines at present with all tests passing.

While I am tempted to spend some time shaving this yak, reducing the boilerplate, I think it’s best to move on to the final exit type.

Falling through pits is half-working. New animations and yet-more-complications of the ExitSystem have gotten me to “falling into a pit”, “spawning a new room”, “fading the old out”, and “falling from the ceiling”.

As you can see, there’s still some jank to it. The camera jumps around a bit, and the animations aren’t quite incorrect. While the player doesn’t appear to be in the pit during the first half of the transition, this is actually correct – pits in the game have surrounding assets that extend the black a bit, and accordingly the player’s falling sprites are bigger than the true pit tile.

I have been able to reuse a lot of the infrastructure I built up for the other exits (things like fading and explicit camera control) which is nice, but I’m still not super pleased with the code. Even before fixing the last few things with pits the ExitSystem has 3 separate code paths, 14 different state variables, and totals more than 1,000 lines of code. I doubt I’ll get around to cleaning it up during my sabbatical, since it’s so close to ending, but if I continue this project I’ll definitely want to revisit the ExitSystem once it’s feature complete.

Continue onto November 27th

Sabbatical Log: November 25th

This blog series is running about a week behind my actual work, giving me time to clean things up.  The date the actual work was done is in the title, horizontal lines indicate when I stopped writing and started coding.

Time to get to work on stairs.

First step is to actually make the entities and get the exit logic working.

I’ve managed to get all the actual loading parts of stairs working, and have reworked system enabling/disabling so animations can play during any exit transition.

Actually building the stairs animations is rather error prone, at the moment I’ve got the first half of going up stairs working alright. The rest will have to wait for later.

I also need to make some changes to rendering to make sure multi-level entities (like the door frames) render appropriately.

Now all animations for stairs are working.

Some interesting things that had to be solved:

• Stairs are necessarily wider than the player, so positioning pre- and post-transition isn’t trivial
• The transition time (that controls the fade in/out) has to be synced to both the player’s animations and their position
• Player’s have to be nudged out of doors so their first post-transition move doesn’t immediately trigger the stairs again
• Subsets of the player control and animation systems have to be enabled during a transition

I’m not in love with the code I’ve written, but it will do for now. There’s one more exit type – pits – I want to tackle, but I need a break from fiddling with sprites and animations. I’m going to detour into component reuse / object pools for a bit, and come back to the final exit type.

Continue onto November 26th

Sabbatical Log: November 24th

This blog series is running about a week behind my actual work, giving me time to clean things up.  The date the actual work was done is in the title, horizontal lines indicate when I stopped writing and started coding.

Busy social day, so I’ll probably only get a little bit done on the new exit types. We’ll see.

I’ve got the basic infrastructure for going through a door setup.

A door is composed to two entities, a frame and the actual door, in a manner similar to a player being composed of three entities (feet, body, and head). As with the player’s feet, the door’s state is owned by just one of the entities (the actual door).

The door’s state is composed of just the target room and the tile coordinates to come out of. I’ve yet to actually implement the transition, but the idea is fade out while the player is still in the first room and fade back in once they are in the new room.

Door transitions are now complete.

The basic idea is quite similar to scrolling transitions, there’s a Step counter, a call from the FrameStateManager to get how much to fade, a new room loaded, and old entities released. I wouldn’t call it the cleanest code, but it works well enough – especially with a gin and tonic in hand.

Stair-style transitions should be almost identical, except a special animation will play. At the moment the animation system is disabled during an exit transition (you can see this by watching the flower fade in on the right in the gif above), so that’ll require some work. The various hitmaps also need some work, as you can see the “pushing” animation happens on the door frame and it’s quite easy to get stuck on the door’s edges.

Tomorrow is the last day of the holiday weekends, so I don’t know how much coding time I’ll get – but I should be able to make some progress.

Continue onto November 25th

Sabbatical Log: November 22nd and 23rd

This blog series is running about a week behind my actual work, giving me time to clean things up.  The date the actual work was done is in the title, horizontal lines indicate when I stopped writing and started coding.

The 22nd was the Thanksgiving in the US – I didn’t get much done.

Back at it on the 23rd, post-holiday. Probably still a light day, but I’m going to work on smoothing out room transitions in the cardinal directions. With any luck, I’ll get to the stair-style transitions.

I’ve got the cardinal directions working now.

If you watch the transitions, you’ll notice the interactions of collision detection and exit systems. An exit placing the player inside a tree (for example) will result in them getting pushed back into an exit zone. I like that the interaction here “makes sense” and is an emergent property of the two systems interacting, not an explicitly handled edge case.

That said, I don’t love the ExitSystem code. It explicitly calculates the desired end position of the player and camera, and takes over camera positioning during an exit transition. I’m going to spend some time making it less ugly before moving onto other sorts of exits.

I’ve now cleaned up the ExitSystem code, and written a few tests. I mostly added comments, collapsed primitive fields into structured types, and moved the camera-y bits over to the CameraSystem. The CameraSystem now has a notion of being “explicitly pointed” somewhere, rather than deferring to another system it still keeps track of the camera but allows other systems to make camera requests. I also moved most of the logic that fills out the FrameState during a transition into the ExitSystem, since it was mostly duplicated anyway.

The ExitSystem remains kind of big and complex, but it’s much more manageable now. I may need another refactoring pass after adding other sorts of transitions, but I’m fine with it for now.

Here’s an outline of the current code:

Next up are the other transitions, for now I’m going to confine myself to four: going through doors, going up stairs, going down stairs, and falling down holes. It makes the most sense to me to model these as colliding with particular objects, as opposed to the edge triggered-ness of the cardinal direction transitions.

Continue onto November 24th

Sabbatical Log: November 21st

This blog series is running about a week behind my actual work, giving me time to clean things up.  The date the actual work was done is in the title, horizontal lines indicate when I stopped writing and started coding.

I’ve done a little more cleanup work this morning. I’ve removed entities from the BushSystem (so it no longer needs special compacting), and I’ve moved the compact calls into a new CleanupSystem. I like having cleanup in a system of its own, since then I get the same profiling as all other systems.

I’ve also decided that, at least for now, I’m going to trigger entity compacting every 100 GameState advances, when the ratio of dead entities to live one is >= 2, or when the EntityManager reaches the end of it’s backing buffer. Triggering any of these conditions resets the iteration count, so logically this means we’ll compact at least once every 100 iterations but may compact more frequently.

Reliably compacting entity storage also let me change the NewEntity() method into a simple bump allocator, so that’s nice.

I’ve decided today that I’m going to work on room transitions.

A random aside, I threw a memory profiler on a DEBUG build out of curiosity. It’s nice to see that there’s basically no churn once things settled down after app start.

At the moment, the only actual allocations are creating new Components. At some point I’ll move those to an object pool, and then we should get to actually zero allocations during normal operation.

I’ve now got the very basics of edge-triggered screen transitions working. There’s still a lot to do, for one the camera isn’t really aware of transitions which results in it lurching forward at the start.

The approach I went with was:

1. Mark everything in the current room as needing to be culled
   1. This uses a FlagComponent, so it’s cheap
2. Load up the new room
3. Go over all the old entities and update their positions so they are in the same logical position, but now relative to the new room’s coordinates
4. Disable loads of systems that shouldn’t be running during the transition
5. Apply a velocity to the player in the appropriate direction, to keep them moving into the new room
6. Let the game state advance until the camera lies entirely within the new room
7. Go remove all the entities marked in step 1
8. Re-enable all the systems disabled in step 4

This approach is pretty generic in terms of keeping things where they “belong” during a transition, and involves a minimum of new code. The only tricky part was getting the math down for transitioning between the rooms – I’m not convinced it’s completely correct.

I’ve been fighting a headache for an hour or so, so I’m going to call it for today. Tomorrow I’ll focus on fixing up the camera’s behavior, making sure transitions work between rooms of different sizes, and getting some test cases in.

Here’s the current behavior.

If you pay attention to the trees you can see the old entities getting culled at the end of the transition. The camera jump is apparent as well – it’s a consequence of the camera always wanting the player to be in the center of the screen, unless at a room’s edge, and the logical removal of aforementioned room’s edge.

Continue onto November 22nd and 23rd

Sabbatical Log: November 20th

This blog series is running about a week behind my actual work, giving me time to clean things up.  The date the actual work was done is in the title, horizontal lines indicate when I stopped writing and started coding.

Today my goal is to get entity management cleaned up, in particular I want to: get traversal proportional to number of entities with some property, compact entity lists, and speed up retrieving components for an entity.

Time to get to work.

I’ve got traversal down, and have removed lots of casts and other nastiness.

I’ve basically added component-type-specific containers to the EntityManager class, so lookups are cheap, and switched enumerations over to intrusive linked lists.

Previously I stored all stateful components in a 2-dimensional array and kept each component type in a specific column – this required some multiplications (and not power-of-two ones) and down casting, plus the ergonomics of multi-dimensional arrays in C# aren’t great. Now every component type gets its own array and accessor methods so there’s no down casting, access is cheaper, and the code is nicer.

Most systems want to iterate over all the entities with a specific component. Prior to this change I was looping over a column in that 2-dimensional array, stopping when I got to the next allocated Entity. This meant a lot of time was spent iterating over empty values, since most components are relatively sparse. Switching to a linked list makes iteration proportional to the number of active components.

I have one small twist on intrusive collections, which is rather that storing pointers I store indexes. Since Entity ids are already used to quickly access components for specific Entities, this has a nice symmetry and (at least in theory) means the GC has less to do. The interface for my linked lists is below:

As with most everything else, the size of the linked list is fixed at app start.

I also took some time to apply the same quick-check optimization I have in the collision detection system to the update position system (which is responsible for making sure entities don’t overlap at the end of a “frame”).

Next up, I need to implement compaction in the entity manager. Time to get back to it.

I’ve got compaction working by all appearances, and I have bunch of tests for it too. The basic idea is that, upon request, the entity manager will scan it’s internal table and renumber entities such that they’re all contiguous. It then takes that remapping, and informs each other thing that cares about entities – letting them update themselves.

Most of the work was punching the appropriate interface into the right places, the actually algorithms are not complicated.

Any instance that holds onto an Entity for longer than one update needs to implement IHoldsEntity (and be passed to Compact). I’m pretty sure I could remove all “extra-EntityManager” entity storage from, and may do so in the future. Right now there are only two classes that do so, the BushSystem (for tracking what is “pending being cut”) and the actual GameState class which has convenience accessors for the parts of a player and the camera entity.

Right now, since I’m debugging, I force a compaction between each system being invoked during an update. In the future I’ll want compaction to happen at more particular points, like screen transitions, and in response to extreme fragmentation.

All said I’m pretty happy with today’s results, I’ll probably do some more cleanup tomorrow and then move onto room transitions.

Continue onto November 21st