This week we produced something that actually looked like an effortful demo from the tool!!

The result - "Almost Infinite" by Logicoma, placing 2nd in the combined demo compo at Solskogen 2016!

And you can grab the binary, read the comments, etc over at pouet.

And this marks a pretty big milestone in the latest iteration of the tooling and Logicoma as a group :)

Because of this, I'd like to do a slightly more detailed writeup about it, or at least some of the larger parts of it. So let's dig in!

Tooling

I decided I wanted to talk about tooling first, as that kindof sets the stage for how the rest of the parts of the demo went (except perhaps the soundtrack). So first, a bit of history.

At last year's Solskogen, h0ffman and I threw together a little 40k called Backscatter. The whole thing was done in something like 12 days from scratch, having only a synth, a "framework" that didn't build and a sync tool. But no matter - we pulled through, and ended up winning the intro compo, which was pretty great! Not to mention I was quite proud of the soundtrack and the fact it came from my softsynth; it was about time this got some more use in an actual prod with visuals :)

However, I was extremely disappointed with what we brought to the table visually. While the code was clean (you can actually find a version of the main shaders on shadertoy) and it ran pretty well (for a raymarcher), everything was a bit too simplistic, felt outdated, and I hated how the only way I felt like I could make it look good was to blow out the colors and add straight lines. Not to mention it ran like shit on both of my high-end laptops, while most modern games of the era don't (Mirror's Edge Catalyst, Battlefront, you name it).

And even before this intro, I was feeling generally the same sentiment - I had never really invested in doing a proper polygon renderer with HDR and all the bells and whistles before, and was generally sick of throwing together everything at the party. I really wanted a set of tools that would actually survive between prods and would make it fun to do visuals again, and have decent perf on the machines I'm typically working with.

First Generation

So, I got to work on some new tooling. I knew I'd want some sort of scene graph representation with nodes and properties that I could tweak in real-time, with file watching for assets and everything. I also knew I wanted it to be cross-platform, as I'm usually on laptops and more consistently running OS X these days than I used to. So, for the first gen of the tool, I decided to use F#/OpenTK/WinForms.

This process generally went pretty well -- development time was very fast, and I had a proof of concept and some nice abstractions within a couple weeks. From then on, I spent some time trying to make some basic stuff, and started this everyweeks project in order to be able to document everything and keep myself (and my buddies) accountable on a week-to-week basis. So, unsurprisingly, many of my first posts here were engine/tooling tests, such as:

• Adultery
• Delft
• Depth of Failure

While these weren't produced entirely with the tool, they were very valuable tests and awesome to work on. However, a huge flaw in the tool design reared its ugly head already during these early stages -- WinForms was SUPER unstable under Mono. Essentially, what was supposed to be a cross-platform tool really only ended up being a Windows-only tool, and that didn't work. Compound that with the fact that F# really isn't that comfortable to work with outside of Windows (unless you're comfortably with MonoDevelop, and honestly, who is), and we're left with a big problem.

So, the tooling went on a bit of an unplanned hiatus, where I worked on other projects (that were super fun, including this site, some emulator-related things in Rust, some hardware/fpga projects, a nostalgia-fueled blitz basic compiler, etc). But fast forward some months to March 2016 or so. Around this time Revision was creeping up, and I was feeling a bit bummed I hadn't made more progress on the demo front for awhile. So, I started looking at more frameworks etc to make a demotool, and ended up choosing Rust/C++/Qt.

This ended up being a pretty solid choice. The cross-platform woes were mostly solved (minus a couple quirks with Qt on OS X), but development was much slower. I think this came down to both the choice of tools (Rust/C++ aren't terrible, but are quite verbose), the fact that I had a bit less free time during this period than usual, and just plain wanting to do things right.

Most of the high-level development details have also been covered in other posts, so I'll save some details and link those:

• Back on demo stuff again :D
• Even more demo stuff :)

I even managed to squeeze out a quick invitation for Solskogen, although it didn't end up using too much of the tooling:

• Finally, something that looks like a demo
• Well it's an invite at least :)

And after that, more tool dev:

• Slider
• Another short week...

Including this monstrous overview of the tool's architecture (now a bit dated, but generally still applicable):

• Finally, that tool architecture writeup :)

Aaaand some more dev:

• Started on project serialization
• Postoshop
• Back to polygons!
• 2:1

But generally, the tool at this point consists of a viewport, a tree view for a scene graph, and an inspector to change properties on the nodes in the graph. Couple that with some decent abstractions for textures, render targets, samplers, etc, and you have yourself an OK tool for authoring. Or well, in theory you do; in practice it wasn't the greatest.

The biggest problem at this point was that nodes were written in Rust, which meant you could basically do anything, but it also meant that you'd have to recompile the tool whenever you added/changed a node. Also, the property system I designed was pretty verbose, so making new nodes was pretty annoying, and I hadn't really solved a way to package nodes properly yet, so all the nodes were stuffed into one huge .rs file. While this ended up working out in the end, it felt horrible and generally it just wasn't nice to work on.

Add to the mix the fact that I also didn't really finish the tree view editing functionality in time (to the point where I implemented these features Thursday before the party, and they were pervasive and caused large refactorings last-minute) and it all felt like a bit mess to be honest.

That said, there were times when this tool REALLY shined - in particular, the actual scenes (minus the one I was using to test the DOF) ended up being doing in 2-3 hours on Saturday morning at the party place. Having a proper editor to tweak values and move things around is totally a gamechanger if you're used to hardcoding things, and proof enough that all of this will be worth it, once we've iterated enough on the design etc. And for that, I've got some tricks up my sleeve for next week's writeup, so stay tuned :)

Anyways, the tool was helpful, but not finished enough, and fundamentally needs a bit of a redesign. But it did the trick for this prod, and that's pretty rad :) So now onto some other parts of the production!

Soundtrack

Coming into this demo, I had originally thought the soundtrack was basically a solved problem. I had done an ambient/IDM track around March or so that I was pretty happy with, and I figured it would go pretty well with some nice, clean visuals. However, it became quite apparent that I didn't have the time budget to actually make the renderer as clean as I wanted, and I had half of a drum and bass drop lying around, so I enlisted Martin to collaborate on making the track a real thing and making it awesome. Over the course of a few evenings this took shape fairly smoothly. We typically collaborate well musically, and this track was no exception, and I think the result was pretty nice.

As far as details in the production, there really isn't much to say other than there was basically almost zero bus processing; everything was just super clean and straightforward and LOUD, which was the point :) and with that, we had a nice and dirty soundtrack for what would be our not-as-clean-as-I-had-hoped visuals :)

Rendering

For rendering, there ended up not being much to it. There's the typical stuff - HDR with proper filmic tonemapping/gamma, DOF, and Bloom, and that's about all I had time to do (because the DOF implementation took ages primarily, and still isn't even done :P). While most of these things are fairly straightforward and it's easy to find literature for them, I do want to elaborate a bit on the DOF specifically.

DOF

DOF is a surprisingly complicated effect to do as a post-process step if you want it to be fast and actually look good. It's especially hard if you're a bit out-of-the-game in terms of watching talks/reading papers/slides and have to acclimate yourself to the lingo a bit, but luckily, that's something I find pretty fun when I can do it on my own time and really dig in.

Now I've already put out two posts talking about this, but they don't go into too much detail:

• Depth of Failure II
• Depth of Failure III

But it's of course more fun and useful to actually include those details, so let's cover some of that :)

At the highest level, you have some "focal plane" where everything in your image should be in focus. Things on that plane will be perfectly sharp, whereas things further away from that plane (either closer or further from the viewer) will be blurrier. While that sounds simple, and can be implemented fairly easily, doing it well is really, really hard.

The method I've been implementing is mostly outlined in Crytek Graphics Gems pages 33-35. Now the actual implementation I have currently isn't entirely similar, as there are some things I've left out for simplicity as well as thins I haven't gotten to yet, but it does work and other than being pretty flickery did the trick for this demo.

The basic idea (again minus too many details from the slides, mostly because they can be confusing if you don't know why they're doing some things) is to split the image into two fields; near and far. The far field represents stuff behind the focal plane - further away from the in-focus stuff. This is the easier field to deal with, as stuff just kinda blends all over the place here (with some caveats ofc, but more on that in a sec). Then we have the near field, which represents stuff in front of the focal plane - closer than the in-focus stuff. This field is a bit harder to deal with; I'll get to that too in a sec.

Basically what we're going to do is mask out bits of the image that are going to be part of the near/far fields into separate images, process those separately, and composite them back together with our original sharp image from the rendering process. This is where things start to get interesting.

Take, for example, this image:

Here we can actually see both fields - the background is the far field, the girl's face is generally in focus (technically only the intersection of her face and the focal plane will be entirely in focus and the rest will be in either of the other fields, but that's not very important for understanding this), and her hat is in the near field.

Notice how the background is very blurry, but that blur never "bleeds into" the in-focus region and the near field. The opposite is true for the near field (though it isn't shown very well in this photograph, except perhaps on the edge of her hat) -- the near field will get very blurry, and will bleed onto everything behind it.

What this mostly means for us implementing the DOF is that we'll be treating our near/far fields differently to handle the different kinds of bleeding when we composite them back together.

For the far field, this is generally pretty easy. For this we'll just do a fun-shaped blur (in my implementation it's a mix between a perfect circle and an n-gon with parameters to tweak) and not include samples that aren't also in the far field. We'll scale the kernel of this blur with a given pixel's "circle of confusion", which is basically the circle around the pixel in which other pixels will contribute to the blur, and it's determined by the pixel's depth. Combine that with some artifact-hiding pre-multiplies and masks (outlined in the crytek slides) and you've got a pretty good-looking far field; not much more to it (actually there is, but it all comes down to speed/quality tradeoffs/optimizations, all of which are outlined in the crytek paper and I did them more or less the same).

For the near field, things are a bit more tricky. Generally, we want to be really careful with how we're doing the color bleeding and masking when we're done. The general premise of "taking a fun blur kernel, scaling by CoC and gathering" is generally the same, excep to make things more accurate, we won't be gathering. We'll be using a scatter-as-gather approximation instead, which is something more like "look at all the surrounding pixels in the max CoC and see if their CoC means they would've contributed to us if they were scattering, and weight their contribution accordingly". I'd love to go into more detail about this, but frankly I'm getting a bit tired of talking about this one part of the demo, and I should really outline it in more detail on its own when my implementation is a bit more finished and I'm more happy with it. But the last think I'll add about this layer is that in my implementation at least, this layer not only keeps track of color information, but has an alpha mask as well that gets blurred too. This is so that bleeding can occur properly in the compositing step.

Once we've processed our separate fields, it's time to put them back together. For the far field, this is pretty easy - just blend between the far field pixels and the in-focus original pixels based on the pixel's far CoC. And as long as this mix isn't linear, it can look pretty good (again, see the crytek slides for this).

For the near field, because we've kept track of our alpha mask, we're going to use that for blending primarily. Now, a small disclaimer: this is part of my implementation that I haven't spent enough time with, and I'm not that thrilled with how mine looks yet, so take that with a grain of salt. But generally we're going to take the near CoC and the blurred alpha mask and use those to mix the out-of-focus near field with the blended near field from before, and with some hacks to hide the fact that we don't actually have any information about what's behind some of the blurry stuff in the near field, Bob's your uncle.

Again, I'm getting a bit sick of spending time writing about this in the context of the whole demo, and my implementation isn't finished, but I wanted to outline at least generally what I'm doing here, as it was one of the primary things that gave the demo its characteristic look.

Visuals

Now it's time to talk about what we actually pumped through the renderer. This was actually the simplest part. Every scene was drawn with exactly one cube -- and a bunch of replications/transforms of course :)

The primary replicators I made were a random replicator, which would make x instances of a given subtree with specified random param's for translation, rotation, scale, and time, and a grid replicator, which makes x by y instances of a subtree with similar parameters (minus time). Making scenes like this is actually really fun - it's awesome to tweak and see how things change, and it's really cool to play with, but of course, the goal is to make it look like it's NOT just all cubes. In this demo, I'm not sure if that was achieved, but at least some of the scenes are pretty cool-looking :)

In terms of other things like lighting, it's basically faked phong simulating a single directional light source and a ton of pow's and stuff to blow it out. It's really not very technically interesting, but it does the trick for making things look dirty (that and some distortion fx in the pipeline to add some color). This was an area where I really would've liked to do more, but because I didn't have time to add proper lights or materials in the tooling, it was one of those things where we just decided grayscale would be the look, and to try and make that look as rad as possible. And to be honest, those kinds of limits are why I love demoscene and making demos - it really gets the creative juices flowing, and I had loads of fun making these scenes, even though they were done in an extremely short amount of time (literally 2-3 hours at the partyplace minus the spiral scene I tested the DOF with)!

As for the main title logo, that was also done in a pinch by Martin, and I think it looks damn sexy. He also went an gathered a bunch of symbols/sci-fi looking decals we thought we'd throw in there, but we simply didn't have time to do it. Perhaps they'll pop up in a future production :)

Direction/Sync

This is the area of the production I'm by far the least happy with, but there are aspects about it that were pretty cool depending on your perspective.

To start - I had zero time to integrate a sync system into the tooling demo last-minute, so we went with a two-part system that was "good enough":

• Make a scene graph node that just displays a different child based on time (and those times were completely hardcoded from music timestamps)
• Add flashes/fades/logo displays in a pixel shader (again with hardcoded times from the music)

This is one of the reasons there aren't as many details in the sync as I usually do, and generally I feel like the sync was a lot weaker than in past productions. But what I absolutely loved about doing it this way was just how well Martin and I worked together to make it happen.

The first part of this was when I was adding BASS integration into the replayer so we could actually play sound (and of course this was Saturday afternoon at the party, a few hours before deadline :P). At this point, we parallelized a bit, and Martin went through the soundtrack marking where all the timestamps for major fades/flashes and scene changes would be. After that, we went through the list and decided which scenes would go where pretty quickly, along with whether or not the scene would be mirrored in the x/y directions or not. After that was a "collaborative data entry" task where he would read off timestamps and the type of sync and I would enter, and it actually felt pretty rad to collaborate this way (not to mention it was surprisingly efficient given what we were doing and had to work with).

The entire sync process took a couple hours (and pushed us over the deadline, but way less so than usual :P), and given the constraints it's something I think we can be proud of, but the lesson really is to make sure your tool has a plan for sync earlier on than uh... at the party :D

Final Thoughts

This demo is pretty special to me for a few reasons. The first is that it's the first sign of anything that actually looks proper after spending much of the last year on updating my graphics programming knowledge and making tools. The second is that because everything happened so late time-wise, and we hit so many last-minute roadblocks, the fact that this thing got released at all was the product of pure tenacity and good collaboration, which is something I'm extremely proud of. This would not have happened at all without Martin's help, and we both learned a bunch in the process, and that's pretty cool. And the last reason this demo is pretty special to me is because it's an actual thing made by the actual tool that I've actually been working on for a proper amount of time. We're not where we want/need to be techwise, but after a few years of "just shitting out demos at the partyplace and nothing more" (minus my SNES escapades ofc) it feels pretty good to properly invest in this and start seeing the benefits. And I think we're really on a roll now too; you'll read about that more in my next post :)

Anyways, Solskogen was a great party; quick shoutout to everyone that was there and everyone who made it special (way too many to name)! Also huge shoutout to some of my really close friends who had more to do with this than they realize, in particular Martin, baggers, zach, rick, aske, and some others - you guys rule :)

And with that, I'll leave you with some various random screenshots and things. Enjoy! :)

Last Edited on Mon Jul 25 2016 11:07:07 GMT-0400 (EDT)