On Configuring Development Tools K. Richard Pixley Cygnus Support Last Mod Tue Oct 1 21:20:21 PDT 1991, by rich@cygnus.com INTRO ----- This document attempts to describe the general concepts behind configuration of the Cygnus Support release of the GNU Development Tools. It also discusses common usage. Eventually, FIXME, there will also be a man page for "configure", an "info" tree, etc. BASICS ------ Some Basic Terms: There are a lot of terms that are frequently used when discussing development tools. Most of the common terms have been used for several different concepts such that their meanings have become ambiguous to the point of being confusing. Typically, we only guess at their meanings from context and we frequently guess wrong. This document uses very few terms by comparison. The intent is to make the concepts as clear as possible in order to convey the usage and intent of these tools. "Programs" run on "machines". Programs are very nearly always written in "source". Programs are "built" from source. "Compilation" is a process that is frequently, but not always, used when building programs. Host Environments: In this document, the word "host" refers to the environment in which this source will be compiled. "host" and "host name" have nothing to do with the proper name of your host, like "ucbvax", "prep.ai.mit.edu" or "att.com". Instead they refer to things like "sun4" and "dec3100". Forget for a moment that this particular directory of source is the source for a development environment. Instead, pretend that it is the source for a simpler, more mundane, application, say, a desk calculator. Source that can be compiled in more than one environment, generally needs to be set up for each environment explicitly. Here we refer to that process as configuration. That is, we configure the source for a host. For example, if we wanted to configure our mythical desk calculator to compile on a SparcStation, we might configure for host sun4. With our configuration system: cd desk-calculator ; ./configure sun4 does the trick. "configure" is a shell script that sets up Makefiles, subdirectories, and symbolic links appropriate for compiling the source on a sun4. The "host" environment does not necessarily refer to the machine on which the tools are built. It is possible to provide a sun3 development environment on a sun4. If we wanted to use a cross compiler on the sun4 to build a program intended to be run on a sun3, we would configure the source for sun3. cd desk-calculator ; ./configure sun3 The fact that we are actually building the program on a sun4 makes no difference if the sun3 cross compiler presents an environment that looks like a sun3 from the point of view of the desk calculator source code. Specifically, the environment is a sun3 environment if the header files, predefined symbols, and libraries appear as they do on a sun3. Nor does the host environment refer to the the machine on which the program to be built will run. It is possible to provide a sun3 emulation environment on a sun4 such that programs built in a sun3 development environment actually run on the sun4. Host environment simply refers to the environment in which the program will be built from the source. Configuration Time Options: Many programs have compile time options. That is, features of the program that are either compiled into the program or not based on a choice made by the person who builds the program. We refer to these as "configuration options". For example, our desk calculator might be capable of being compiled into a program that either uses infix notation or postfix as a configuration option. For a sun3, chosing infix might be: ./configure sun3 +notation=infix while a sun4 with postfix might be: ./configure sun4 +notation=postfix If we wanted to build both at the same time, in the same directory structure, the intermediate pieces used in the build process must be kept separate. ./configure sun4 +subdirs +notation=postfix ./configure sun3 +subdirs +notation=infix will create subdirectories for the intermediate pieces of the sun4 and sun3 configurations. This is necessary as previous systems were only capable of one configuration at a time. A second configuration overwrote the first. We've chosen to retain this behaviour so the "+subdirs" configuration option is necessary to get the new behaviour. The order of the arguments doesn't matter. There should be exactly one argument without a leading '+' sign and that argument will be assumed to be the host name. From here on the examples will assume that you want to build the tools "in place" and won't show the "+subdirs" option, but remember that it is available. In order to actually install the program, the configuration system needs to know where you would like the program installed. The default location is /usr/local. We refer to this location as $(destdir). All user visible programs will be installed in $(destdir)/bin. All other programs and files will be installed in a subdirectory of $(destdir)/lib. You can elect to change $(destdir) only as a configuration time option. ./configure sun4 +notation=postfix +destdir=/local Will configure the source such that: make install will put it's programs in /local/bin and /local/lib/gcc. If you change $(destdir) after building the source, you will need to: make clean before the change will be propogated properly. This is because some tools need to know the locations of other tools. With these concepts in mind, we can drop the desk calculator and move on to the application that resides in these directories, namely, the source to a development environment. SPECIFICS --------- The GNU Development Tools can be built on a wide variety of hosts. So, of course, they must be configured. Like the last example, ./configure sun4 +destdir=/local ./configure sun3 +destdir=/local will configure the source to be built in subdirectories, in order to keep the intermediate pieces separate, and to be installed in /local. When built with suitable development environments, these will be native tools. We'll explain the term "native" later. BUILDING DEVELOPMENT ENVIRONMENTS --------------------------------- The Cygnus Support GNU development tools can not only be built with a number of host development environments, they can also be configured to create a number of different development environments on each of those hosts. We refer to a specific development environment created as a "target". That is, the word "target" refers to the development environment produced by compiling this source and installing the resulting programs. For the Cygnus Support GNU development tools, the default target is the same as the host. That is, the development environment produced is intended to be compatible with the environment used to build the tools. In the example above, we created two configurations, one for sun4 and one for sun3. The first configuration is expecting to be built in a sun4 development environment, to create a sun4 development environment. It doesn't necessarily need to be built on a sun4 if a sun4 development environment is available elsewhere. Likewise, if the available sun4 development environment produces executables intended for something other than sun4, then the development environment built from this sun4 configuration will run on something other than a sun4. From the point of view of the configuration system and the GNU development tools source, this doesn't matter. What matters is that they will be built in a sun4 environment. Similarly, the second configuration given above is expecting to be built in a sun3 development environment, to create a sun3 development environment. The development environment produced, is a configuration time option, just like $(destdir). ./configure sun4 +destdir=/local +target=sun3 ./configure sun3 +destdir=/local +target=sun4 In this example, like before, we create two configurations. The first is intended to be built in a sun4 environment, in subdirectories, to be installed in /local. The second is intended to be built in a sun3 environment, in subdirectories, to be installed in /local. Unlike the previous example, the first configuration will produce a sun3 development environment, perhaps even suitable for building the second configuration. Likewise, the second configuration will produce a sun4 development environment, perhaps even suitable for building the first configuration. The development environment used to build these configurations will determine the machines on which the resulting development environments can be used. A WALK THROUGH -------------- Native Development Environments: Let us assume for a moment that you have a sun4 and that with your sun4 you received a development environment. This development environment is intended to be run on your sun4 to build programs that can be run on your sun4. You could, for instance, run this development environment on your sun4 to build our example desk calculator program. You could then run the desk calculator program on your sun4. The resulting desk calculator program is referred to as a "native" program. The development environment itself is composed of native programs that, when run, build other native programs. Any other program is referred to as "foreign". Programs intended for other machines are foreign programs. This type of development environment, which is by far the most common, is refered to as "native". That is, a native development environment runs on some machine to build programs for that same machine. The process of using a native development environment to build native programs is called a "native" build. ./configure sun4 Will configure this source such that when built in a sun4 development environment, with a development environment that builds programs intended to be run on sun4 machines, the programs built will be native programs and the resulting development environment will be a native development environment. The development system that came with your sun4 is one such environment. Using it to build the GNU Development Tools is a very common activity and the resulting development environment is very popular. make all will build the tools as configured and will assume that you want to use the native development environment that came with your machine. Using a development environment to build a development environment is called "bootstrapping". The Cygnus Support release of the GNU Development Tools is capable of bootstrapping itself. This is a very powerful feature that we'll return to later. For now, let's pretend that you used the native development environment that came with your sun4 to bootstrap the Cygnus Support release and let's call the new development environment stage1. Why bother? Well, most people find that the Cygnus Support release builds programs that run faster and take up less space than the native development environments that came with their machines. Some people didn't get development environments with their machines and some people just like using the GNU tools better than using other tools. While you're at it, if the GNU tools produce better programs, maybe you should use them to build the GNU tools. It's a good idea, so let's pretend that you do. Let's call the new development environment stage2. So far you've built a development environment, stage1, and you've used stage1 to build a new, faster and smaller development environment, stage2, but you haven't run any of the programs that the GNU tools have built. You really don't yet know if these tools work. Do you have any programs built with the GNU tools? Yes, you do. stage2. What does that program do? It builds programs. Ok, do you have any source handy to build into a program? Yes, you do. The GNU tools themselves. In fact, if you use stage2 to build the GNU tools again the resulting programs should be identical to stage2. Let's pretend that you do and call the new development environment stage3. You've just completed what's called a "three stage boot". You now have a small, fast, somewhat tested, development environment. make bootstrap will do a three stage boot across all tools and will compare stage2 to stage3 and complain if they are not identical. Once built, make install will install the development environment in the default location or in $(destdir) if you specified an alternate when you configured. In fact, you can skip the "make all" part and just "make install" which will make sure that the development environment is built before attempting to install anything. Even better, for configurations where host is the same as target, like this one, "make install" will make sure that a "make bootstrap" is done before installing anything. Any development environment that is not a native development environment is refered to as a "cross" development environment. There are many different types of cross development environments but most fall into one of FIXME basic categories. Emulation Environments: The first category of cross development environment is called "emulation". There are two primary types of emulation, but both types result in programs that run on the native host. The first type is "software emulation". This form of cross development environment involves a native program that when run on the native host, is capable of interpreting, and in most aspects running, a program intended for some other machine. This technique is typically used when the other machine is either too expensive, too slow, too fast, or not available, perhaps because it hasn't yet been built. The native, interpreting program is called a "software emulator". The GNU Development Tools do not currently include any software emulators. Some do exist and the GNU Development Tools can be configured to create simple cross development environments for with these emulators. More on this later. The second type of emulation is when source intended for some other development environment is built into a program intended for the native host. The concept of universes in operating systems and hosted operating systems are two such development environments. The Cygnus Support Release of the GNU Development Tools can be configured for one such emulation at this time. ./configure sun4 +ansi will configure the source such that when built in a sun4 development environment the resulting development environment is capable of building sun4 programs from strictly conforming ANSI X3J11 C source. Remember that the environment used to build the tools determines the machine on which this tools will run, so the resulting programs aren't necessarily intended to run on a sun4, although they usually are. Also note that the source for the GNU tools is not strictly conforming ANSI source so this configuration cannot be used to bootstrap the GNU tools. Simple Cross Environments: ./configure sun4 +target=a29k will configure the tools such that when compiled in a sun4 development environment the resulting development environment can be used to create programs intended for an a29k. Again, this does not necessarily mean that the new development environment can be run on a sun4. That would depend on the development environment used to build these tools. Earlier you saw how to configure the tools to build a native development environment, that is, a development environment that runs on your sun4 and builds programs for your sun4. Let's pretend that you use stage3 to build this simple cross configuration and let's call the new development environment gcc-a29k. Remember that this is a native build. Gcc-a29k is a collection of native programs intended to run on your sun4. That's what stage3 builds, programs for your sun4. Gcc-a29k represents an a29k development environment that builds programs intended to run on an a29k. But, remember, gcc-a29k runs on your sun4. Programs built with gcc-a29k will run on your sun4 only with the help of an appropriate software emulator. Building gcc-a29k is also a bootstrap but of a slightly different sort. We call gcc-a29k a simple cross environment and using gcc-a29k to build a program intended for a29k is called "crossing to" a29k. Simple cross environments are the second category of cross development environments. Crossing Into Targets: ./configure a29k +target=a29k will configure the tools such that when compiled in an a29k development environment, the resulting development environment can be used to create programs intended for an a29k. Again, this does not necessarily mean that the new development environment can be run on an a29k. That would depend on the development environment used to build these tools. If you've been following along this walk through, then you've already built an a29k environment, namely gcc-a29k. Let's pretend you use gcc-a29k to build the current configuration. Gcc-a29k builds programs intended for the a29k so the new development environment will be intended for use on an a29k. That is, this new gcc consists of programs that are foreign to your sun4. They cannot be run on your sun4. The process of building this configuration is another a bootstrap. This bootstrap is also a cross to a29k. Because this type of build is both a bootstrap and a cross to a29k, it is sometimes referred to as a "cross into" a29k. This new development environment isn't really a cross development environment at all. It is intended to run on an a29k to produce programs for an a29k. You'll remember that this makes it, by definition, an a29k native compiler. "Crossing into" has been introduced here not because it is a type of cross development environment, but because it is frequently confused one. The process is "a cross" but the resulting development environment is a native development environment. You could not have built this configuration with stage3, because stage3 doesn't provide an a29k environment. Instead it provides a sun4 environment. If you happen to have an a29k lying around, you could now use this fresh development environment on the a29k to three-stage these tools all over again. This process would look just like it did when we built the native sun4 development environment because we would be building another native development environment, this one on a29k. The Three Party Cross: So far you've seen that our development environment source must be configured for a specific host and for a specific target. You've also seen that the resulting development environment depends on the development environment used in the build process. When all four match identically, that is, the configured host, the configured target, the environment presented by the development environment used in the build, and the machine on which the resulting development environment is intended to run, then the new development environment will be a native development environment. When all four match except the configured host, then we can assume that the development environment used in the build is some form of library emulation. When all four match except for the configured target, then the resulting development environment will be a simple cross development environment. When all four match except for the host on which the development environment used in the build runs, the build process is a "cross into" and the resulting development environment will be native to some other machine. Most of the other permutations do exist in some form, but only one more is interesting to the current discussion. ./configure a29k +target=sun3 will configure the tools such that when compiled in an a29k development environment, the resulting development environment can be used to create programs intended for a sun3. Again, this does not necessarily mean that the new development environment can be run on an a29k. That would depend on the development environment used to build these tools. If you are still following along, then you have two a29k development environments, the native development environment that runs on a29k, and the simple cross that runs on your sun4. If you use the a29k native development environment on the a29k, you will be doing the same thing we did a while back, namely building a simple cross from a29k to sun3. Let's pretend that instead, you use gcc-a29k, the simple cross development environment that runs on sun4 but produces programs for a29k. The resulting development environment will run on a29k because that's what gcc-a29k builds, a29k programs. This development environment will produce programs for a sun3 because that is how it was configured. This means that the resulting development environment is a simple cross. There really isn't a common name for this process because very few development environments are capable of being configured this extensively. For the sake of discussion, let's call this process a "three party cross". FINAL NOTES ----------- By "configures", I mean that links, Makefile, .gdbinit, and config.status are built. Configuration is always done from the source directory. * "./configure name" configures this directory, perhaps recursively, for a single host+target pair where the host and target are both "name". If a previous configuration existed, it will be overwritten. * "./configure hostname +target=targetname" configures this directory, perhaps recursively, for a single host+target pair where the host is hostname and target is targetname. If a previous configuration existed, it will be overwritten. * "./configure +subdirs hostname +target=targetname" creates a subdirectories H-hostname and H-hostname/T-targetname and configures H-hostname/T-targetname. For now, makes should be done from H-hostname/T-targetname. "./configure +sub name" works as expected. That is, it creates H-name and H-name/T-name and configures the latter. Hacking configurations: The configure scripts essentially do three things, create subdirectories if appropriate, build a Makefile, and create links to files, all based on and tailored to, a specific host+target pair. The scripts also create a .gdbinit if appropriate but this is not tailored. The Makefile is created by prepending some variable definitions to a Makefile template called Makefile.in and then inserting host and target specific Makefile fragments. The variables are set based on the chosen host+target pair and build style, that is, if you use subdirectories or not. The host and target specific Makefile may or may not exist. If fragments * Makefiles can be edited directly, but those changes will eventually be lost. Changes intended to be permanent for a specific host should be made to the host specific Makefile fragment. This should be in ./config/hmake-host if it exists. Changes intended to be permanent for a specific target should be made to the target specific Makefile fragment. This should be in ./config/tmake-target if it exists. Changes intended to be permanent for the directory should be made in Makefile.in. To propogate changes to any of these, either use "make Makefile" or re-configure from the source directory. * configure can be edited directly, but those changes will eventually be lost. Changes intended to be permanent for a specific directory should be made to configure.in. Changes intended to be permanent for all configure scripts should be made to configure.template. Propogating changes to configure.in requires the presence of configure.template which normally resides in the uppermost directory you received. To propogate changes to either configure.template or a configure.in, use "configure +template=pathtothetemplate". This will configure the configure scripts themselves, recursively if appropriate. * "./configure -srcdir=foo" is not supported yet. At the moment, things will probably be configured correctly only for leaf directories, and even they will not have paths to libraries set properly.