Alright so when we left off last week I was saying that there was still more speed to be gotten out of the data conversion routines.

The speed cost comes from the fact that when we are writing lisp data into foreign memory we are calling a generic function to do it. A generic function in lisp is kind of like a virtual method in other languages but with a couple of differences. It uses what is known as multiple-dispatch (methods like this are often called multimethods)

So in languages like C# and Java, methods live in classes.

class Foo
{
    void Bar() { .. }
}

.. and when you call someVar.Bar() the way it picks which version of the method to call based on the type of someVar. That's why this is called single-dispatch, it only looks at one 'parameter' (yup the thing before the dot is though of as an parameter in this case)

In multiple dispatch it picks version of the method to call based on the type of all of the arguments. This is a really cool feature (it makes the visitor pattern redundant for example), however complexity is never free and so it makes it slightly more expensive to work out which method to use (trivia: this is known as calculating the effective method)

Now in most programs we don't care, lisp's OOP system does this plenty fast enough and has sensible caching to make things plenty fast enough for most systems. However, we are making things that will live in the heart of tight loops, and as such we would like to avoid the cost.

That was a bit of a detour, but now we are going to avoid that dispatch. Lisp's FFI (foreign function interface) has this nice feature that you can tell it "for some foreign type x here is the code you can inject inline to to the translation". Then whenever it sees you reading from foreign memory and it knows the type, it will inject the custom translation code.

That feature is cool but for foreign structs it only had a version for reading and not writing. So I added it.

I have a pull-request waiting on that project now so hopefully next month that feature will be shipping.

With that I was able to optimize the write conversion further which took a nice chunk out of the load put on the GC. It is good enough for now and I will leave it as it is until it becomes a problem again.

Document some of the things

CEPL finally shipped this week WOO! it is now available in lisp by typing (ql:quickload :cepl) in the repl.

This was awesome, and then I saw this article by another lisper. The TLDR is that lisp folks don't write enough documentation for their projects and it hurts the adoptability of the language. It's true, and I'm rather guilty of that myself. So I resolved to start my api documentation in ernest.

I found the dude who wrote the article and asked him how he normally generates his documentation. Turns out he has his own library for generating docs from the documentation strings lisp lets you add to functions. The html it spits out is sensible enough and easily changed via a template if needed.... it's also ugly as balls.

I have roped in the help of my lovely wife, who has much better taste than I and she is working on the design, css & other stuff that will make this thing tasty.

Inline GLSL

One request I had from a user a while ago was to enable raw glsl to be used in CEPL. I added that feature this week.

Here is the simplest possible fragment shader in CEPL, it simply a passthrough:

(defun-g frag ((color :vec4))
  color)

With the new feature you can write this:

(def-glsl-stage frag-glsl (("color_in" :vec4) &context :330 :fragment)
  "void main() {
       color_out = color_in;
   }"
  (("color_out" :vec4)))

This can be used in CEPL's shader pipelines just like any other gpu function, and you can recompile this whilst CEPL is running and the glsl is uploaded immediately. You will see that you don't write the in or out declarations yourself, you give CEPL that information in a more lispy form. This means that CEPL doesn’t have to analyze the glsl to know how to connect this to other shaders in the pipeline and also means you don't have to have the names of the in vars be the same as the out vars of the previous stage.

This was actually a bit of a PITA to get right. Normally you have to have the in and out vars names and types match between shader stages, but I remembered some feature where you could set the glsl location and use that instead. It would be PERFECT.

I spent a good couple of hours reading the GLSL spec, collating all the details and I was just finishing up by looking at the GL Wiki and I saw this

Program Separation Core in version: 4.5 Core since version: 4.1

SHIT.

shit shit shit.

I need to support v3.3 and up so all that time was wasted. Shit.

So in the end what I do is make the names match but also add a global var that with the name you actually wanted. So for the example def-glsl-stage a couple of paragraphs back I generate:

#version 330

in vec4 OUT_43; // <-- this matches the previous stage.

vec4 color_in = OUT_43; <-- and this is what we wanted.

layout(location = 0) out vec4 color_out;

void main() {
    color_out = color_in;
}

So yay! features from v4.1 in v3.3, DONE.

Repeat after me "I will not add geometry shaders, I will not add geometry shaders, .."

I keep deciding I will add geometry shaders as it shouldnt be too much work, but every time I realise that there is so much other stuff to do on CEPL that is way more valuable. This week was no expection.

The problem is that it would be a kickass feature to have, and to implement, but it is SOO not critical right now when I could be fixing bugs or writing docs.

Ah bugs, CEPL does have plenty of them. I killed a few of those this week, including three that touched the flow analyzer I was working on over christmas. It was cool looking at that again.

That's all folks

That's plenty for one week,

Good-night all!

Last Edited on Sun Mar 20 2016 19:32:07 GMT-0400 (EDT)