Making "Ad Astra" (1/3)

This is a short behind the scenes about "Ad Astra" , a demo for the Pico-8 virtual console that I worked on recently with ilkke  Here's a video of it, of you can watch it online.

I got into Pico-8 coding after seeing some short-form gif tutorials that the author, Lexaloffe, uploaded to his Twitter account.  At the time I was on the tail end of a large commercial game project, so it was a nice distraction writing in this self-contained development environment after work.  Being an occasional demo coder the first things I worked on were replicating a few popular demoscene effects: the Plasma, Vertical Rasters and then Vector Bobs.

After doing a few of these independently of each other I thought about some way to release them, and that's when I decided to try making a little demo 'engine' to run these scenes in sequence, with maybe some sort of design to tie them together.

Demo Engines

In the modern demoscene a demo engine is a broad term, in it's basic form an engine runs behind the scenes, managing the sequence of demo parts, assets, handling their set up and the overall timing.  On the other end of the scale you have fully integrated tool suites like Werkkzeug, which was the production tool behind a lot of Farbrauch's content like the famous fr-08 from a few years back.

What I wanted to do was something that handled the back-end work and left me free to create scenes from the various effects I'd made easily.  This boiled down to three requirements:

• Each part of the demo can be timed accurately.
• Each part can be made from modular components so they can be re-used, and the 'engine' will handle initialisation and constructing the draw and update loops for it.
• The content of scenes can be mostly constructed outside Pico-8 for easy iteration using a very basic script language, making it a "data-driven" demo rather than hard-coding each screen to fit.

So let's see how each point was tackled:

1) Timing

The first thing I needed to do was work out a way to do timing.  Not timing for effects, we have _draw() and _update() for that, but general timing of the demo flow.  Modern demos usually sync to the music as it's a constant and there is likely an audio system in place providing rock-solid timing for you.  In Pico-8 there are a couple of stat() variables you can use to find out which pattern a channel is playing (16-19), and what position it's got to in that pattern. (20-23)  There aren't any for song position so the way I did it was to check if we'd hit pattern position 0 and used a flag to check if we were still in the same pattern.  I decided that each scene in the demo could be X song patterns in length, so when the timing flag is set it decreases this counter by one and if it reaches zero we know it's time to move on to the next screen.

2) Modular Components

As with games, modern demos (as in PC) are usually coded out of modular components.  This is so scenes can be constructed from various re-usable parts to make a greater whole and saves on code duplication.

Let's take an example, here's a scroller screen I coded beforehand.  It has a starfield, vector bobs, the scroll and a mirror effect.  They're all hardcoded to that one screen.

Instead you split each element out into functions and give them as many input variables as you can, without impacting on performance too much.  This instantly makes each thing more flexible and gives the designer a lot more scope to construct scenes from these smaller parts to make a larger whole.  It also applies to effects you're only going to use once, like the landscape part in Ad Astra, because you can enhance these scenes with other modules you've already written.

So now each effect had three functions: Init, Draw and Update.  Actually some of them didn't have an update because I did that in the Display bit instead (naughty) but never mind.  That's another rule of making demos: if it seems to be faster do it that way instead.

When a new scene starts the engine checks which modules are used and feeds their Init function with the relevent data for that scene.   It then builds array lists for the _draw() and _update() functions.  These are basically just loops checking if a value is true in the array and calling the relevant effect function at that point.

The z-order of things in a scene is decided by their order in the _draw() array.  This was good for backgrounds and overlays where needed.

Effects aren't all truly modular as some of them only have one instance of their variables available (the Bobs & Vectors for a start) , however the important bits that get re-used often (like the Map module) are setup properly and can be re-used multiple times in a scene.

3) Building scenes outside Pico-8

This was mostly about making things more comfortable for the creative process.  The easier it is to make your demos the faster you can iterate and improve them, you're more likely to do that in a workflow that isn't awkward.

To that end I made a few tools to help out, these were all made in the old Blitz Basic which, for me at least, I still find the fastest way to make little graphics tools.  As only the two of us were ever going to use them they didn't need to look nice, they just had to work.

The main tool was both a script compiler and .p8 file builder.  You feed it a script in the right format which it turns into an array the demo engine can read.  It then splices this with a .p8 file of the demo engine and exports a new compiled version.  Finally it boots pico8 and runs the file automatically for convenience. 

Here's a quick example of the really simple script format.  First two lines are the p8 filename to splice into (useful for checking earlier versions if something starts breaking) and the timing values for each scene in array format.  The 255 at the end is a special command to tell it to loop on the last scene.

demoengine_v46.p8
{1,10,6,5,4,4,1,3,7,7,9,5,1,255}

Then each part follows, this is a list of the modules used in the Z order we want them to be displayed.  Each module starts with a # followed by the name and then the data for that part afterwards.  For example:

#map
80 ; layerxmap
4 ; layerymap
0 ; layerxpos
47 ; layerypos
16 ; layerxsize
12 ; layerysize
0 ; layerxposreset
0 ; layeryposreset
0 ; layerxspeed
7 ; layeryspeed
0 ; layerxposadd
-1 ; layeryposadd
0 ; layerxmapadd
0 ; layerymapadd
0 ; layerxposmax
0 ; layeryposmax
#vector
0 ; skip mesh reset?
50 ; start zoom
148 ; xpos offset
48 ; ypos offset
0.5 ; x rotate start
0.0 ; y rotate start
0.0 ; z rotate start
0.000 ; x rotate move
-0.001 ; y rotate move
0.0003 ; z rotate move
700 ; target zoom
4 ; zoom speed
-1.1 ; xpos add
0.15 ; ypos add
1 ; mesh object
#end

There are a few extra commands to handle adding data (such as the 3d meshes) and using Pico-8 commands within a scene, like the palette controls.   This might seem odd but because it's data-driven we want to avoid hard coding things per scene if possible.  So while changing the palette through an extra function rather than directly may lose a tiny amount of cpu time it means it's available in the script with the other parts of the scene. There were several times where I had to change masking colours for particular map layers and being able to do that in one continuous process made things much easier.

Each scene ends with an #end command so the engine knows we're done.

This may not look like a much faster way of working, but being able to run this in Notepad++ , tweak a setting here and there, cut/paste entire parts around to change the flow etc. certainly helped with the production workflow.

Another useful tool I made was a .PNG to _map & _gfx converter for the artwork.  This was like any old tile converter in that it optimized the tiles used and then reconstructed the .png in the new order as a _map & _gfx set that could copy/pasted into the .p8 file.  So ilkke could send me his latest version of the artwork as a .PNG file, made in whatever he wanted to use:

And I then had these in the engine ready to view straight way.

The final tool I did was a converter for .OBJ (Wavefront) files to turn them into an array for the vector routine.  The only easy way to store vertex colours was in materials, so we did that.  Here's a screenshot of ilkke's original Blender spaceship:

Hindsight

In hindsight my use of an array meant I wasted a LOT of tokens, and didn't throughly check that out until we started running out of memory.  Though luckily it was only in the last days of production.  Next time I'll use a pure bitstream approach and also standardize the input format for parts so we don't have any fixed variable names.

In the next two posts I'll go through each part in turn with a bit more detail.

Making "Ad Astra" (2/3)

Scene 1 : Flyby

The opening scene makes the most use of the Map module, this is a little wrapper around the Map function that adds variables for movement over time, and checking boundries for resetting the map's position.  By stacking these up I could have some parallax layers to set the scene.  The stars use the Vector Bob module on a mesh of random X,Y,Z points, with a very slight rotation on one axis so it's direction gradually moves from head-on to the side-view. 

ilkke supplied some great assets for this, a full-size space background across several maps, the detailed space freighter and finally our Ate Bit logo.  This is also the first time you see the triangular design style ilkke went with on the backgrounds to each scene.

If you're unfamiliar with the term Vector Bobs, it's basically a mesh of points sorted in Z order and rendered with sprites.  To add more depth you can use different sized sprites based on the Z ordering.  I think the first time a lot of us saw this effect was the Afterburner arcade game's title screen.

In the early days of the Amiga demoscene it was quite a popular effect, probably the most famous is in the Red Sector Megademo.

Scene 2: Space Flight

So now we use the Vector Bobs again with classic bubble sprites, and bring in a filled spaceship to fly over it while a planet hovers into view behind.

Amusingly this is the first software triangle routine I've ever done, so it was fun looking up old tutorials to find out how to best approach it.   Ones I used were this for the triangle splitting algorithm and this old one from Cubic & $eeN on backface culling.  I did end up with a couple of bugs in the draw which were temporarily fixed with Rectfill to cover them up.  I did mean to go back and fix it properly but I was running out of time so that stayed in, it didn't seem to impact the speed much.  Anyway, something to work on next time.

For sorting both bobs and tris I used a Comb Sort, I think I did a literal translation of the psuedo code from here in the end, short and fast enough for what I needed.  I did have a faster version in the demo which shared the Z position and index in one token, it had a hard-limit of 511 items (not really a problem for the scenes I wanted to do) but had some problems with really close Z positions that I didn't notice until late in the day.  As I didn't have time to fix that up and the scenes were still in a frame without it I swapped back to the older two token version instead.   The one in the demo puts out about 150 bobs in a frame, the faster one was hitting about 190.   Here's a gif of the fast one at full capacity, just under 3 frames sorting 511 bobs:

Scene 3 : Scroller

This keeps the previous elements and adds a sine scrolling logo over the top.

The logo is one long map, when displayed the visible area is split into vertical strips of 1x16 in size, and each one of these has an independent Y sine position which is stored in an array.  Strips are pixel scrolled left, and once 8 pixels have been covered the pixel position is reset and the X position of the map draw increases by one so that the map scrolls across the screen.  

To do the sine effect the last element in the Y array has a new sine position calculated, then the whole array is shifted to the left every frame so it's constantly Y disting across the whole visible map.

Scene 4 : Planetfall

This uses the map and vector modules again (actually all screens use the Map module in some way), the only new thing here is the Plasma module which is used for the cloud effect.

The first effect I coded on Pico-8 was a standard Plasma, to make things a little smoother I did it as a 4x4 pixel effect in sprites, plotting the sprites for the map every 4 pixels in X + Y direction:

The Plasma module has a few extra inputs for max sprites to use, area and also what resolution to use.  Amusingly while it can still do the 4x4 mode I only use it in normal 8x8 sprite resolution in the demo.

Scene 5 : Landscape

The landscape is in the style of classic 'voxel map' effects like the famous Mars demo from 1993.  Obviously it's a lot less capable than any of those but it does use a height-map (displayed in sprites so I could have a bit of a gradient effect with the lighting) but can only move in map shifts, no rotation or camera movement is possible.   The Z distance calculation is far more like one you'd do for a classic racing game effect than proper 3d too.

The lighting is calculated based on the sun sprite's X + Y position relative to the displayed sprite position. But it's one of those 'demoscene calculations' where you tweak it to look alright rather than basing it on any real-world formulas.  As we're not shifting the map position left or right it's actually mirrored in the middle to save a bit of cpu time, but with slightly different lighting calculations for each half so it's not immediately obvious.

To finish the scene off I asked ilkke to do a cockpit overlay (which looks great), this is shifted up and down on the same sine calculation as the sun sprite.  In the background of the scene a couple of Rectfills draw the sky and horizon, which also moves on the sun's sine wave.

This is the only effect that goes over a 'frame', or at least goes over the stat(1) output whch is what I was using to test each screen during development.

I put an earlier version of this scene up on my twitter (below), but I think the new version looks a bit with the gradient sprites.  Still lots of room for improvement though.

Making "Ad Astra" (3/3)

Scene 6 : Ship flyover

This re-uses most of what we've already seen, with the only new effect being the Water module which is only 5 lines of code.  This uses memcpy to copy part of the screen line by line to another part of the screen but in reverse order.  To get the ripple effect I run a sine calculation in the copy loop and add the result to the line to copy, the start positions of the sine calculation are stored outside the loop and updated on each new call. This way you get the distortion of lines being copied out of order, but because the sine is starting from a moving constant you get a gradual movement of the distortion vertically up the area, making it seem much more like waves.   After this point I use it in pretty much all the scenes because it was fast and was a nice way to fill the spare space with action.

The Plasma module is used again here to add some subtle ripples on the water surface. I'm quite happy with how that turned out because with the small window size it creates some quite diverse shapes in the water.  With ilkke's great map layer work behind and in front of the water I think this was quite effective in the end.

Scene 7 : Shaded vectorbobs

Here we use the Vector Bobs again on a 5x5 cube.  The lighting is again calculated by the bob's X+Y position against the Sun sprite's X+Y position.  It averages this out so that the X + Y results are always a positive.  If both X+Y results are under a certain threshold (a simple way of saying the bob is close enough to the sun to be lit) then we do some division on both values to end up with a final result that is somewhere in the spectrum of bob sprites in the bank.  I think we had 8 lighting values in the end and as long as they're in the right sprite order bobs closest to the sun will get the brightest colours.

To give the calculation a bit more of a 3d appearance the X + Y are divided by different amounts (half for Y) and I use the bob's Z position as a divider to give it more of a spread across the cube surface.   These are, of course, 'demoscene calculations' in that you keep tweaking until you get what you want for the effect.

Anyway I was quite happy with this one, that lighting spread was my main aim and it gives the right appearance even if the maths are very far from a proper lighting model.

Scene 8 : Vertical Rasters

Another oldskool effect, I was quite happy to see that this can be done in nearly the same way I've done it on 8-bit machines before.  Basically you have a 1-pixel high horizontal strip which is plotted onto each vertical line of the screen.  For each vertical line you plot into it your vertical bar graphic at a sine calculation.  You then update the sine calculation, wait for the next vertical line and plot your bar graphic into it again.  You continue doing this for the whole screen and because you're plotting into the same pixel line of graphics without clearing it you end up with the 3d snakey effect above.  

In the pico-8 version I have a one-pixel high strip of sprites which is plotted into as above and then I use the map command to redraw it one pixel down each time.  Simpler than doing it on a Vic-20 obviously. :) 

To enhance it a bit I use the Map X position to give me some wobbly sine offsets, and I do the loop in reverse order so it comes out upside down.   That's very easy on here where you're not chasing the raster beam down the screen, but it's also a homage to the one in LDA STAin by Clarence/Chorus which was a complete headscratcher for me when it came out.   He then went one better in Rocketry by drawing them in reverse Z order, amazing stuff.

Finally as I'm plotting into a map I can make use of transparency and have some bubble sprites behind the rasters to add depth, the horizon also bobs up and down using Rectfill and the raster sine position calculation.  Everything else in this scene is handled by the Map module and Water so I can hide the logo behind it when I need to.

Scene 9 : Blenk vectors

Classic blenk vectors, I think the first of these I saw was in Enigma by Phenomena though those were centrally lit.  This uses no new modules but adds a moving sun sprite.  The lighting calculation is much simpler for this and only uses the sun's X position relative to each triangle's X position.  There is no threshold check, it just divides down to a nicely useable number for an array of the lighting colours: darkest to brightest. 

So that's all the vector routines in the demo.  One I didn't get to use was an extra mode which combined wireframe with filled to have a psuedo-drawn effect.  The wireframe has a random offset from the origin each frame to make it look more sketched.  As it didn't fit in to the design I didn't remove the triangle joins which would make this look neater.

Scene 10 : Mirror bobs

So here is the final screen, this almost didn't get in because we'd started running out of memory in the blenk vector part.  To claw it back I turned all the init functions into one big unwieldy if..else list, generalised some variables and finally hard-coded some movement routines only to the parts they're used in. (like the shade bobs and blenk vector)

In this part the vector bobs are plotted to the same origin position on an empty screen, which is then copied to a 4x4 block of sprites in one of the banks.  The screen is then cleared and 4 map strips are drawn onto the screen next to each other, these strips contain the same sprites we just plotted into, so we get a mirror copy of the vector bobs all over the screen.  To add variety the map strips are offset by 1 y map position each and are longer than the display so they can be shifted vertically and then reset every 4 map blocks for a seamless moving background.  I put an earlier version of this on my twitter which got quite a few likes, so thanks for that:

  
As there was some cpu time left I put the logo scroller over it for the finale.  I think this part is still running in a frame, stat(1) says yes but it does look like it staggers occasionally, maybe that's just the draw update.

Transitions

In most scenes there's a transition at the start and end, this just uses Rectfill and each of the X+Y Start and End points can have a number added or subtracted to give different fills and reveals. (with different colours)  I hard-coded the timing for when these start and end (start is literally at the start of a new screen, end is a few beats into the last pattern of the part) which worked ok here but isn't that flexible obviously.

Music

One rule of thumb for doing music for demos (and trailers too) is to have the music first, and time your scenes to it.  But personally if I'm coding the demo as well the last thing I'm thinking about is writing the music.  One obvious flaw with writing it last is you are stuck to the timing of the demo which usually won't be as bar friendly as you'd like.   You'll notice the music in the demo doesn't really follow much of a song structure because the scenes aren't very even in length.

One thing I will say though is make use of soundfx in your music.  Having the ship engine whooshes and odd portamento effects really helped with dynamics. (One thing I should have put in was a 'Shake' module for shifting the camera around, but I got around some of that using sound fx)  This is still something that doesn't happen very often even in modern PC demos and I don't really understand why.

End

Thanks for reading, hope you enjoyed watching the demo.  Massive thanks to ilkke for all his support working on this.