Layout Engine, Entity System, and an Epic Fail.

Stas Lisetsky  —  3 weeks, 6 days ago
The heart of Cascade it it's layout engine - the second most complicated thing in the app. (Scale tool has first place so far.)
I'd never written a UI engine before, but I decided not to read anything on the topic and just go in blind, relying on intuition only. The plan was to get the 'feel' for how things work and only then start the reaserch.
And after more than one year I'm still not there. We've build a quick prototype in about 3 months, and after that there hasn't really been much UI work. Just tools, workflow and internal systems. As well as lots and lots of learning.

So anyway. As a start I took a website of a certain design studio and replicated it in C. After maybe 3 passes I had a working engine that supported rows, columns, auto-wrapping, printing text and so on.

Nothing too comlicated yet, but enough to start working on the actual app UI. Since then the engine has grown into a somewhat monstorous 1000-line function that does all the layout.

So how it works.
It's something like imgui style UI, but not entirely. (I think?)
The dynamic UIs can be pretty complicated, with objects depending on properties of other objects and so on. So I though at least 2 passes to draw the UI were necessary.
- First you specify things as a tree structure of objects. Like, this object is a row. And let's push some children onto it. And this child is 10% width of the row. And this child has auto-width and stretches etc.
Typical object spec looks like this:
vo *CurtainHandle = PushContainer(&GuiObjectBuffer.Plain, HtmlTreeContainer, V2(CurtainTolerance * 2.0f, 300.0f), RGBA(0, 0, 0, 0));   
CurtainHandle->Interactive = true;
CurtainHandle->Positioning = VOPositioning_Relative;
CurtainHandle->P.X = Gui->CurtainX;
CurtainHandle->Height = DividerHeight;
CurtainHandle->UIID = UIID(&Gui->CurtainX, 0, VORole_HtmlTreeCurtainHandle, "curtain_handle");

- And then shove this tree into the function thet resolves all the dependencies, derives final pixel positions and fills in the vertex/color/uv buffers for the renderer

This has been working pretty well so far. I suspect I'm going to need more passes in the future for more complex dependencies between objects. Like when they are on sepatrate branches in object tree.

Now that we've started talking about objecs, let's discuss 'entity' system. And get closer to the epic fail part.

So the entity system is a giant struct with all possible properties + the header with hierarchy pointers + renderer fields (verts, colors, uvs etc.) It still hass lots of things that I'll have to deal with later (like 12 bools!)

There's one cool thing about it, that happened sort of by itself. My render list is a collection of arrays - one for each shader.
struct push_buffer_pocket {
    uint32 EntryCount;
    memory_slice Slice;
    push_buffer_entry *Entries;

struct push_buffer {
    push_buffer_pocket Plain;
    push_buffer_pocket Textured;
    push_buffer_pocket Glyphs;
    push_buffer_pocket Transparent;

But each render object is part of an engine tree as well. So no matter where in the hierarchy the object is, it's going to be a part of linear array when it comes to rendering.
And it's easy to change what shader is going to be used.
I have no idea if this is actually good, but it just felt good to see this data structure flexibility naturally emerge.


Now. The fun part.

At some point I noticed that my object struct 'vo' (visual object) - got to ~1kb. So 1kb per every single thing on the screen. Even 1x1px static pixel - 1kb.
It this bad? Sounds like it's bad, right? My answer now is - I don't know, probably. But before I arrived to this conclusion I went through hell and back.

One day I sat down and decided it was time to make things cool. There'll be different types of objects. They'll all have same header so I can still traverse them both linearly and as a tree.
And since there were lots and lots of object properties and they could be arbitrarily enabled/disabled, I decided to use flag-based types.
How that would work - at the time of creation, I specify which properties an object is going to have - inner offsets, outer offsets, different positioning types and so on.

All in all this was a cool exercise. I was feeling smart, there were some serious macros involved. I have begun considering meta-programs etc.
But after around 5 days of updating all objects and all UI specification in a 15KLOC program - I was feeling awful. it was disgusting drudge work. I was physically ill from ripping the guts out of my app. (I even had to start using accessor fuctions, instead of directly working with struct fields!)
The list of errors was not getting smaller, I was fixing issue after issue. And then a question popped into my mind - 'Why am I doing this again?'. Then I stopped and stared thinking.

When I cant find a logical answer in my head I turn to numbers. Numbers don't lie. And numbers said that the most this program will ever have - is probably around 2-3 hundred thousand objects (which will be something insanely complex). So even if it's 500K, 500*1k is 500 megs.
Is that a lot? probably, for a simple 2d app (which is supposed to be super-optimized and Handmade in every possible way). But that's not the point.
The point was - this is the only number I had. My UI was like 100-200 objects tops, not counting glyphs. It was impossible to justify any sort of major refactoring or optimization, let alone this massive re-write.

And maybe I was right, maybe this was a good idea and I'll end up having to do this again. But not until I have a valid necessity, supported by numbers. Actual stats: which properties do I use the most. Is it even worth to do this type system? Maybe it'll be used just to support 0.001% of all objects or something - and be a complete waste of time and lead to unnecessary increase in code complexity.

So then I did the only sane thing - I reverted all the changes. And was the happiest man alive, because my program actually compiled and ran. And there was nothing wrong with it.
It took another week to recover from that psychologically, and I'm not even joking. I'm trying not to be attached to code and erase/rewrite things when I feel something's not working, and in this case I just got impatient, and paid the price. I still learend a lot though. Now I know how I'm going to write this type-based system if I need it in the future.

So that's that.

Next time we'll talk about the two-faced god Janus.

Oh, and have some more debug art. This time it's just bug art, actually.
(I changed something accidentally and Bruce from previous post got spread out in a cool-looking wavy line.

Log in to comment