Humanizing the Build Process with SCons

Yes, writing build files can be as much fun as coding in Python.

Atul Varma

Humanized, Inc.

The Complexities of Building Software

• Build only what's needed
• If a source file changed since the last build, rebuild only the target files that depended on that source file.
• Build systems don't really apply to situations where source files don't need to be built into target files (e.g., Python).
• Decoupled from system environment
• Builds need to be consistently reproducible on a variety of systems, so build files should be independent from the particulars of the system environment they're run on.
• Also useful for building multi-platform software.
• Easy to write, easy to maintain
• Reading build files shouldn't be harder than reading the source code they're building!

GNU Make's Approach

• "Hello World," in a Makefile:



HelloWorld : HelloWorld.c
    gcc HelloWorld.c -o HelloWorld

• It builds only what's needed.
• It violates DRY ("Don't Repeat Yourself").
• It's tightly coupled to the system it runs on.
• Make has no way to automatically determine what files HelloWorld.c depends on.
• If the Makefile itself changes, Make has no way of knowing.

Make is Pretty Low-Level

• GNU Automake and GNU Autoconf allow the developer to have Makefiles generated automatically for them.
• These tools take care of implicit dependencies (e.g., determining what files HelloWorld.c depends on).
• These tools allow the Makefiles to be generated in a platform-independent way, thus decoupling the developer's build files from the system they run on.
• However, there are costs involved in using these tools. For one thing, it means that you ultimately have to learn at least three different tools: Make, Automake, Autoconf, and potentially more.

Where Automake and Autoconf Lead Us.

AM_CHECK_PYTHON_HEADERS(,[AC_MSG_ERROR(could not find Python headers)])

# get rid of the -export-dynamic stuff from the configure flags ...
export_dynamic=`(./libtool --config; echo eval echo \\$export_dynamic_flag_spec) | sh`

# cairo
PKG_CHECK_MODULES(CAIRO, cairo >= cairo_required_version)
if test -n "$export_dynamic"; then
  CAIRO_LIBS=`echo $CAIRO_LIBS | sed -e "s/$export_dynamic//"`
fi

# cairo + cairo-xlib + gtk + pygtk
if test x"$with_pygtk" = xyes; then
  # was cairo compiled with cairo-xlib enabled?
  save_LIBS="$LIBS"
  LIBS="$CAIRO_LIBS"
  AC_CHECK_LIB([cairo], [cairo_xlib_surface_create], [], [with_pygtk=no])
  LIBS="$save_LIBS"
fi

Um...

Sure, that's readable.

Introducing SCons

• "Hello World," in an SConstruct:



Program( source = "HelloWorld.c" )

• It doesn't violate DRY!
• It's easy to read!
• It's completely decoupled from the system environment!
• It automatically figures out what HelloWorld.c depends on!
• SCons knows when SConstruct files are changed!
• It's Python!

Okay, Calm Down.

• Python is imperative, but SCons constructions are declarative.
• This can make things a little confusing at times.

The SCons Builder Object

• Example of Builder Methods:

Program( source = "HelloWorld.c" )
Install( "/usr/bin", "HelloWorld" )
  

• The fundamental SCons component.
• Encapsulates a mechanism that converts given source files into target files.
• Internally, Builders are composed of SCons Action objects that ultimately execute shell statements.
• Internally, Builders can have SCons Scanner objects that automatically search through source or target files to discover additional implicit dependencies (e.g., to find #include'd header files).

Batteries Included!

• SCons comes pre-packaged with Builders for...
• C/C++ programs
• C/C++ static and shared libraries
• Java stuff
• Fortran stuff
• Assembly stuff
• LaTeX documents
• SWIG extensions
• And more!
• It's also not hard to make your own Builder.

The SCons File Node Object

• Example:



myProg = Program( source = "HelloWorld.c" )
print "Building program in %s." % myProg[0].abspath
Install( "/usr/bin", myProg[0] )
  

• Encapsulates any file that's part of the build process.
• Can be passed to any SCons functions or methods instead of a filename string.
• Builder methods always return a list of nodes corresponding to their target files.
• Can be useful when making a build process platform-independent.

The SCons Environment Object

• Example:



myDebugEnv = Environment(
  CPPDEFINES = { "_DEBUG" : "1" }
  )
myDebugEnv.Program( source = "HelloWorld.c" )
myDebugEnv.Program( source = "GoodbyeWorld.c" )
  

• Encapsulates a fully isolated system build environment.
• One SConstruct can have as many environments as it wants.
• Builders are actually contained within Environments; if you don't create an Environment explicitly, an empty one is implicitly created for you.

Some Environment Caveats

• An Environment isn't propogated from the system environment used to run SCons. It's completely independent.
• Environment variables (e.g., CPPDEFINES) aren't the same thing as shell environment variables (e.g., PATH). Shell environment variables are contained in the ENV Environment variable as a dictionary (just like os.environ).
• Environment variables can be used in strings that are processed by SCons by using the format "${MY_VARIABLE}" (as with Make and shell scripts).
• It can sometimes be hard to figure out what strings are or aren't processed by SCons; ones that aren't can be manually processed by using the Environment.subst() method.

Other Cool Stuff SCons Supports...

• Custom command-line arguments
• Separating source and build directories
• Multi-platform configuration (Autoconf functionality)
• Caching built files
• Building from code repositories
• Fetching files from source code management systems
• Parallel builds

Where now?

• Check out www.scons.org.
• Read the User Guide for a thorough overview and tutorial of SCons.
• Use the Man Page as a detailed reference.