# What Comes With Chromium?

This section describes the major components of Chromium and how they fit together.

- The configuration mothership

  The mothership handles all configuration requests for all components of the system. The mothership is written in python and can be found in the mothership/server/mothership.py file.

- Several example configuration scripts

  The mothership/configs directory contains a number of sample configuration scripts. You'll probably want to use one of these scripts as the basis for making your own.

- A "render" SPU

  This SPU hands all OpenGL calls to the system's OpenGL implementation. Nearly every Chromium configuration will use the render SPU (or readback SPU).

- A "pack" SPU

  This SPU packs a stream of OpenGL commands into a buffer which is sent over the network to a network server node. Although the calls are encoded verbatim (i.e., with no analysis), some state tracking is still necessary in order to properly unpack pixel data for texture maps.

- An OpenGL replacement / faker library

  This library "fakes" unmodified applications into using Chromium. The main thing that the faker library does is to load the client-side SPUs and dispatch OpenGL calls to the first SPU in the chain. Although SPUs implement the OpenGL interface, they typically only export a very small number of symbols. The SPU loader will create a collection of function pointers for the SPUs functions. The replacement library is necessary to actually resemble the system's OpenGL interface sufficiently to broker requests from applications to SPUs. Note that an application could easily load a SPU from disk itself and use it without the OpenGL library, which would be one possible way to render to a local screen and over a network simultaneously.

- An application launcher

  The launcher, called crappfaker, launches applications in a special way that causes them to load the OpenGL replacement library rather than the system's library.
  The application launcher is usually refered to as an application node. Application nodes and server/network nodes host SPU chains.

- A network server

  This server, called crserver, is responsible for dispatching incoming blocks of encoded OpenGL commands (such as from the pack SPU) to a chain of SPUs. In fact, as we will see, the network server is really a stream serializer. It is the job of the server to accept streams from multiple clients, and context switch between them as allowed by the parallel API extensions. In addition, a server can manage multiple display tiles, and is responsible for re-decoding a block of work multiple times, with appropriate modifications to the viewing matrices for multiple tiles.
  The crserver is otherwise known as a network or server node.

## Other SPUs

In addition to the SPUs shown so far, several more are provided. Some of the SPUs are intended to be used directly in a configuration graph, and others are more useful as starting points for new SPUs, or SPUs to inherit from.

- The "dist_texture" SPU

  This SPU allows you to have texture images saved to disk on nodes running crserver processes. The textures can later be loaded by specifying only the filename, thus avoiding the network traffic cost of sending the entire image. This is accomplished by intercepting `glTexImage2D` calls and checking the value of the `type` parameter. If it is `GL_TRUE`, then the texture data (in the `pixels` parameter) is assumed to be a `NULL`-terminated filename string followed by texture data of the given width and height (which need not be powers of two). The image data must be interleaved RGB values in `GL_UNSIGNED_BYTE` format and will be written to disk as a raw PPM image. Once the file is written, the image will be padded if necessary (so that the width and height are powers of two) and sent downstream as a valid `glTexImage2D` call. If the `type` parameter is `GL_FALSE`, then the `pixels` parameter is assumed to be the filename of a raw PPM image file which will be read, padded if necessary, and sent downstream as a valid `glTexImage2D` call. This extension to `glTexImage2D` was developed for the NCSA Pixel Blaster (a movie player for tiled displays), which is a good example of its use. Unlike the original NCSA modifications to WireGL, the distributed texture SPU does not ban absolute filenames because Chromium's crserver processes are meant to be started by the same user who is running the application. The original WireGL modifications were by Paul Rajlich, but don't bug him about this version; all the bugs are due to David Thompson.

- The "error" SPU

  This SPU will issue a fatal error on any OpenGL call. The error message will contain the name of the function called. This SPU is useful for handling "impossible" function calls. If the reader is already familiar with SPU inheritance, all SPUs are implicitly a subSPU of the "error" SPU.

- The "feedback" SPU

  This SPU will catch all related Feedback and Selection OpenGL calls. This allows the abstraction of this layer away from other SPUs. If you need support for Feedback and Selection, then insert this SPU into the chain.

- The "motion blur" SPU

  This SPU uses the OpenGL accumulation buffer to simulate motion blur in a dynamic (animated) scene.

- The "nop" SPU

  This SPU will silently discard all OpenGL calls. This SPU is most useful for understanding the performance of applications and cluster configurations. By replacing the render SPU with the nop SPU, time spent actually executing OpenGL commands can be eliminated. Similarly, the nop SPU can be used on the client to make all OpenGL calls "free". This helps establish a baseline for performance.

- The "passthrough" SPU

  This SPU will pass all calls to the next SPU in a chain. Note that it can therefore never appear as the last SPU in a chain (this is a checked, fatal error). This SPU is not useful by itself, but it is a very useful SPU to inherit from in order to implement a subset of the OpenGL API. An example of such a SPU is shown in the "Writing a new SPU" section.

- The "perf" SPU

  This SPU will monitor OpenGL traffic and display 'interesting' results on what needs to be monitored. See the Perf SPU page for more details on its configuration.

- The "print" SPU

  This SPU was mentioned in the "Configuration scripts" section. It produces a human-readable dump of all OpenGL calls, either to standard output (default), or to a file. This is an invaluable tool for debugging, although it does have a fairly serious impact on performance. Some attempt is also made to turn `GLenum` types back into human-readable strings (thanks to Kekoa Proudfoot for this code).

- The "readback" SPU

  This SPU is derived from the render SPU. As such it accepts a stream of OpenGL commands and renders them into its window. When `SwapBuffers` is called, however, the color buffer image is sent to the child SPU via `glDrawPixels`. Optionally, the depth (Z) buffer image is also sent to the child SPU. By sending the color and depth images from several readback SPUs to one render SPU, one can implement a sort-last (Z-compositing) architecture.

- The "saveframe" SPU

  This SPU saves the current frame to a JPEG or PPM file. Just like the print SPU, this SPU can be very useful for debugging streams. It uses its downstream child to do a `ReadPixels`, so this SPU can never be the last one in the chain. (It also might require a render SPU in the same node, though this hasn't been tested.)

- The "template" SPU

  This SPU actually implements no functions (it is therefore exactly equivalent to the SPU, due to the implicit inheritance). However, a script is provided to rename all files and update the source code to create a new SPU of a given name. Therefore, when creating a new SPU, a copy should be made of the spu/template directory, and this script should be run. This is the approach used in the Writing a new SPU section.

- The "tilesort" SPU

  This SPU uses the techniques described in our SuperComputing 2000 paper to drive a tiled display efficiently from a single serial stream of OpenGL commands. One-third of the logic of WireGL can be found in this library (the other thirds being in the server, and the state tracker, listed below).

- The "wet" SPU

  This SPU warps and distorts the rendered image to simulate a watery surface.

## Useful libraries

Chromium also builds several shared libraries which can be used by any component.

- An "application stub" library for Chromium-aware applications

  This library can be used by Chromium-aware applications that wish, for instance, to use the Parallel API. It also provides routines for creating an OpenGL context without ever opening a window, a useful thing for a distributed application to do (since it doesn't necessarily make sense for dozens of blank faked-out client windows to spring up all over the cluster, especially when all of your cluster nodes aren't running X-Windows). This approach is preferred over defining the Chromium-specific functions as OpenGL extensions and using something like `wglGetProcAddress` to extract the function pointers, since the application stub library is portable and applications don't need to concern themselves with the type of system they are running on.

- A client library for the configuration mothership

  Applications that need to talk to the configuration mothership will use this library to make queries. It hides the fact that the mothership is possibly on a remote machine through an opaque `CRConnection` data structure. New SPUs or other system components can access any piece of configuration through this library.

- A packer library for packing OpenGL commands into network buffers

  This library provides a "packing" function for each OpenGL function. When such a function is called, the function arguments and an opcode are packed into a buffer for eventual transmission. Although this library exports a full OpenGL interface, it is not really a SPU. In particular, the packing functions for OpenGL calls requiring pixel data (e.g., `glTexImage2D`) require additional arguments detailing the unpacking state. For an example of this usage, see spu/pack/pack_pixel.c. Currently, this library is used by both the pack and tilesort SPUs.

- A library for loading SPUs from disk

  This library loads a SPU from a named file and creates its dispatch table. Although this library is mostly designed to be used internally by the OpenGL stub library and the server, applications can load SPUs explicitly if they wish.

- A library for tracking OpenGL state

  This library provides the necessary functionality to track the complete OpenGL graphics state. This library is quite complex, both because the OpenGL state is large and intricate, and also because it contains the logic for doing differencing between contexts. Graphics contexts are represented with "hierarchical dirty bits" so that they can be compared extremely efficiently, a frequent operation when updating a tiled display or switching between contexts on the server. This library does most of the OpenGL error checking, and satisfies `glGet*` requests locally. For a description of the techniques used by this library, see our 2000 Eurographics/SIGGRAPH Workshop on Graphics Hardware paper. This library is used heavily by both the tilesort SPU and the server, and the pack SPU uses it sparingly to keep track of the pixel unpack state for textures.

- A library for unpacking OpenGL command buffers

  The dual of the packing library mentioned above, this library decodes network-encoded OpenGL streams and dispatches them to an arbitrary SPU interface. Currently this is only used by the server, although a special SPU could use the pack library and the unpack library together to make multiple passes over a stream.

- A "utility" library

  This library contains a number of useful routines and data structures that are used by many components of the system. A (possibly non-exhaustive) list of features exported by this library is:

  - Buffer pool management -- efficient "allocate" and "free" of fixed-size buffers
  - A DLL abstraction for portably opening shared libraries explicitly and extracting symbols
  - A portable abstraction for interacting with environment variables
  - A set of error reporting/warning/debug routines, and an assertion macro
  - Hashtables
  - A portable heap-based memory allocator that (severely) handles memory allocation errors.
  - Routines for packing/unpacking pixel data
  - Routines for robustly handling strings, including splitting strings on a substring (useful in parsing mothership responses)
  - Routines for cracking apart URLs, since servers are specified using a pseudo-URL type syntax
  - A connection-based networking abstraction supporting multiple transport layers, credit-based flow control, and message fragmentation
  - A "devnull" implementation of the networking abstraction that drops packets on the floor. Useful for measuring the performance of the "pack" or "tilesort" SPUs without involving the actual physical transport medium.
  - A TCP/IP implementation of the networking abstraction
  - A Myrinet GM implementation of the networking abstraction
  - Thread and process management

## Other programs

A few small programs are built as part off the Chromium build process. They won't all be listed here, as many of them are small test or demo programs.

- resetms

  This program resets the mothership to its initial state. Because multiple applications or servers could run on one computer (called, say, "sluff"), the mothership remembers when it has been contacted by a server running on "sluff", so the next time a server running on "sluff" identifies itself, the mothership knows which one it is. This means that after a Chromium run, the mothership is in an unusable state, since it will no longer accept server connections from "sluff". Running resetms will destroy all of the mothership's internal state. This is generally preferable to quitting the mothership and starting it again, especially since the socket isn't always properly released, which will prevent the mothership from starting up on its default port for a minute or two.

- quitms

  This program contacts a running mothership and tells it to quit. Useful in circumstances when you can't easily get to the machine the mothership is running on.

- sputest

  This program is a simple demonstration that an application can load a SPU and use it without the help of crappfaker or the OpenGL stub library. It draws a single triangle rotating around the center of the screen.

- psubmit

  This is just about the simplest example of a parallel OpenGL program. Instance "1" of this program will draw a blue triangle, and instance "2" of this program will draw a green triangle. With the right config script (see mothership/configs/psubmit.conf for an example), this program can be used to verify that the server's parallel API synchronization primitives are working and that parallel submission is, in theory, possible.

Most of the other programs that are built are designed to exercise one particular portion of the system. Some of them are contributed by WireGL users (thanks in particular to Peter Kirchner for providing some VERY stripped down programs that trip a bug), and there are a number of programs that each demonstrate the functionality of a single OpenGL extension, thanks to Chris Niederauer (all found in the progs/extensions/* directories). Run them. Show off anisotropic texture filtering on your display wall, it's fun.

## Miscellany

The Chromium distribution contains some other useful stuff that isn't a SPU, library, or program:

- A pretty slick build system

  Chromium's build system is mainly designed to achieve two goals. First, creating a new Makefile for a library or program should be as simple as possible. Second, it should be possible to build the code on multiple operating systems from the same network-mounted drive without any object files or dependencies conflicting with each other. Users who want to port Chromium to a new system will need to create a new make-system-configuration file in config/, most likely just using one of the existing files as a template. The build will also quit with an error on various system-dependent things in the "utility" library, such as the abstraction for DLLs, etc. The build system also includes a very fast perl-based C/C++ dependency generator due to Matthew Eldridge in scripts/fastdep.pl. This was rewritten in Python by Christopher Waters to remove the Perl requirements from Chromium.

- This documentation

  A copy of this documentation can be found in doc/, if that's not where you're reading it from already.

- A parsed representation of the OpenGL API

  The scripts in glapi_parser/ parse a massaged version of the Windows OpenGL header, and also a Chromium-specific header containing our supported extension functions (e.g., `glBlendColorEXT`, `glBarrierExecCR`, etc). This parsed representation is used by almost every component of the system to automatically generate some or all of their code.

- Useful routines for manipulating the parsed representation of the OpenGL API

  The routines available in opengl_stub/stub_common.py are available for use by any Python script that is generating Chromium code. It is used quite heavily throughout most of the system. In particular, it provides a very easy way to tag certain OpenGL functions as "special". For example, many of the OpenGL packing functions can be auto-generated, but some (like `glMaterialfv`) are more complicated and need to be coded by hand. The complex functions can simply be listed in a separate file, and the contents of that file efficiently examined through the API exposed by these Python routines.