Part 5 continues with a discussion of the essentials of the CQL Runtime. As in the previous sections, the goal here is not to go over every detail but rather to give a sense of how the runtime works in general -- the core strategies and implementation choices -- so that when reading the source you will have an idea how it all hangs together. To accomplish this, we'll illustrate the key pieces that can be customized and we'll discuss some interesting cases.
The parts of the runtime that you can change are in
cqlrt.h, that file invariably ends by including
cqlrt_common.h which are the runtime parts that you shouldn't change. Of course this is open source
so you can change anything, but the common things usually don't need to change --
provide you with everything you need to target new environments.
The compiler itself can be customized see
rt.c to emit different strings to work with your runtime.
This is pretty easy to do without creating a merge hell for yourself. Facebook, for instance, has its
own CQL runtime customized for use on phones that is not open source (and really I don't think anyone
would want it anyway). But the point is that you can make your own. In fact I know of two just within
We'll go over
cqlrt.h bit by bit. Keeping in mind it might change but this is
essentially what's going on. And the essentials don't change very often.
The rest of the system will use these,
cqlrt.h is responsible for bringing in what you need
later, or what
cqlrt_common.h needs on your system.
Contract and Error Macros
CQL has a few different macros it uses for errors.
usually all map to
assert. Note that
tripwire doesn't have to be fatal, it can log
in production and continue. This is a "softer" assertion. Something that you're trying out
that you'd like to be a
contract but maybe there are lingering cases that have to be fixed
The Value Types
You can define these types to be whatever is appropriate on your system. Usually the mapping is pretty obvious.
The Reference Types
The default runtime first defines 4 types of reference objects.
These are the only reference types that CQL creates itself. In
fact CQL doesn't actually create
CQL_C_TYPE_OBJECT but the tests
do. CQL never creates raw object things, only external functions
can do that.
All the reference types are reference counted. So they need a simple shape that allows them to know their own type and have a count. They also have a finalize method to clean up their memory when the count goes to zero.
You get to define
cql_type_ref to be whatever you want.
Whatever you do with the types you'll need to define a retain and release method that uses them as the signature. Normal references should have a generic value comparison and a hash.
Now each of the various kinds of reference types needs an
object which probably includes the base type above. It doesn't
have to. You can arrange for some other universal way to do
these. On iOS these can be easily mapped to
release macros should all map to the same thing.
The compiler emits different variations for readability only. It
doesn't really work if they don't have common retain/release
Boxed statement gets its own implementation, same as object.
Same for blob, and blob has a couple of additional helper macros that are used to get information. Blobs also have hash and equality functions.
Strings are the same as the others but they have many more functions associated with them.
The compiler uses this macro to create a named string literal. You decide how those will be implemented right here.
Strings get assorted comparison and hashing functions. Note blob also had a hash.
Strings can be converted from their reference form to standard C form. These macros define how this is done. Note that temporary allocations are possible here but the standard implementation does not actually need to do an alloc. It stores UTF8 in the string pointer so it's ready to go.
The macros for result sets have somewhat less flexibility. The main thing
that you can do here is add additional fields to the "meta" structure. It
needs those key fields because it is created by the compiler. However the
API is used to create a result set so that can be any object you like. It
only has to respond to the
Those can be mapped as you desire. In principle there could have been
a macro to create the "meta" as well (a PR for this is welcome) but it's
really a pain for not much benefit. The advantage of defining your own "meta"
is that you can use it to add additional custom APIs to your result set that
might need some storage.
The additional API
is used in the event that you are moving ownership of the buffers from
out of CQL's universe. So like maybe JNI is absorbing the result, or
Objective C is absorbing the result. The default implementation is a no-op.
The CQL run test needs to do some mocking. This bit is here for that test. If you
want to use the run test with your version of
cqlrt you'll need to define a
sqlite3_step that can be intercepted. This probably isn't going to come up.
If you want to support profiling you can implement
to do whatever you want. The CRC uniquely identifies a procedure (you can log that). The
index provides you with a place to store something that you can use as a handle in
your logging system. Typically an integer. This lets you assign indices to the procedures
you actually saw in any given run and then log them or something like that. No data
about parameters is provided, this is deliberate.
The definitions in
cqlrt_common.c can provide codegen than either has generic
"getters" for each column type (useful for JNI) or produces a unique getter that isn't
shared. The rowset metadata will include the values for
CQL_NO_GETTERS is 0. Getters are a little slower for C but give you a small number
of functions that need to have JNI if you are targeting Java.
Encoding of Sensitive Columns
By setting an attribute on any procedure that produces a result set you can have the selected sensitive values encoded. If this happens CQL first asks for the encoder and then calls the encode methods passing in the encoder. These aren't meant to be cryptograhically secure but rather to provide some ability to prevent mistakes. If you opt in, sensitive values have to be deliberately decoded and that provides an audit trail.
The default implementation of all this is a no-op.
You must provide helpers to "box" and "unbox" a SQLite statement
cql_ref_type of the appropriate type. These are used
to create boxed cursors.
The Common Headers
The standard APIs all build on the above, so they should be included last.
Now in some cases the signature of the things you provide in
cqlrt.h is basically fixed,
so it seems like it would be easier to move the prototpyes into
However, in many cases additional things are needed like
other system specific things. The result is that
cqlrt.h is maybe a bit more
verbose that it strictly needs to be. Also some versions of cqlrt.h choose to
implement some of the APIs as macros...