|
|
发表于 2003-4-8 17:15:49
|
显示全部楼层
从字面意思看是：纯LFS。这里有篇文章自己看一下吧
TITLE: How to build a "pure" LFS
LFS VERSION: CVS
AUTHOR: Ryan Oliver <Ryan.Oliver@pha.com.au>,
Greg Schafer <gschafer@zip.com.au>
SYNOPSIS:
The current LFS build method is a lot better than it used to be but is
still somewhat flawed in that not enough emphasis is placed on building a
correct toolchain. This was recently highlighted by criticism levelled at LFS
from senior Linux kernel hacker Alan Cox on the kernel mailing list. Here we
present a new strategy aimed at building a correct toolchain and thus a "pure"
LFS system. It is our belief that building a correct toolchain is the single
most important lesson that people reading the LFS book need to know. Everything
else is secondary.
HINT:
CHANGELOG
=========
* 2003-XX-XX Initial revision
INTRODUCTION
============
Broadly speaking, the toolchain comprises binutils, gcc and glibc. As of this
writing, the current LFS version is 4.0. The problems with the build method as
used in this version can be summarised as follows:-
* too reliant on the host (build) system
e.g. the autoconf tests for the static tools are performed whilst looking at
the host system which may be vastly different from the final system. The
most widely documented and glaring manifestation of this problem is gcc
which ends up miscompiling a portion of glibc due to this flaw (the
HAVE_GAS_HIDDEN hack works around this particular problem).
* too much code is brought in from the host system
e.g. all the static tools are statically linked against the host's glibc.
The host's glibc is an unknown quantity and may contain bugs or anything.
* too susceptible to incompatible glibc changes
e.g. statically linked binaries compiled on a glibc-2.2.x based system that
contain calls to getpwuid() crash when run on a glibc-2.3.x based system.
* glibc is compiled whilst no existing glibc is present
e.g. the glibc autoconf tests produce different results depending on whether
an existing glibc is found or not. This leads to inconsistencies and
encourages hacks and workarounds.
(NOTE - some of the abovementioned issues have been partially addressed in the
CVS version of the book but still do not go far enough and leave too much room
for error.)
To overcome all the issues, a new approach is required. The essence of what is
presented below can be summarised as follows:-
* build a fully "self contained" and "self hosting" toolchain that has no
dependencies on the host system.
The key differences between this approach and current LFS are:-
* a glibc is built in Chapter 5
* this glibc is built using the Chapter 5 gcc and binutils (NOT the versions
from the host)
* the rest of Chapter 5 is built against the new glibc using the new binutils
and gcc
* Chapter 5 packages are installed into a prefix of "/stage1" rather than
"$LFS/static"
Some of the advantages to doing it this way are:-
* we can build everything shared. It isn't necessary that we build all our
binaries static in Chapter 5 (though you still can if you want to) as we
will still have access to the shared libs we create when we chroot.
* everything we use to build our shared binaries and libraries in Chapter 6
will ALREADY be built and linked against the glibc we are migrating to. This
avoids glibc migration issues now and in the future.
The key technical points of how the new approach works are:-
* careful manipulation of ld's linker scripts (in particular, the SEARCH_DIR
variables)
* patching of gcc's "specs" file to tell gcc which dynamic linker to produce
executables for
* clever use of mount's "--bind" option to play a few filesystem tricks
(NOTE - use of mount's "--bind" option mandates a minimum kernel version of
2.4.0 on the host. If the host's kernel is earlier than 2.4, an alternative
"symlink hack" method can be employed. But "mount --bind" is definitely the
neatest and preferred solution.)
At this point, it is worth mentioning there are other, less sophisticated ways
of attaining a "pure" LFS. A "brute force" method is to build Chapter 6 twice.
i.e. remove the /static directory (both physically and from the `PATH') then use
Chapter 6 to rebuild itself. This is the basic idea behind the method employed
by the Linux Standards Base folk with their LSB Sample Implementation[1]. Taking
this approach is not practical for the LFS book. It is far too laborious and
nowhere near as educational with regard to some of the toolchain internals we
describe below.
PROCEDURE
=========
Toolchain cleanliness
---------------------
It is important to realise there are varying degrees of "anal retentiveness"
we can employ when building a new toolchain. The more anal we are, the "cleaner"
the toolchain will be. For this reason it might seem we are building packages
more times than we really need to, but you have to keep in mind, the goal here
is to have a fully self hosted and very clean toolchain in which to perform the
final clean build of our target system in Chapter 6. Steps that are not strictly
necessary (i.e. less anal) will be pointed out as we move through the build. In
other words, a few shortcuts can be taken at the expense of anal retentiveness.
How we tested
-------------
We started from a glibc-2.2.5/gcc-2.95.3 based host to ensure any glitches were
caught early. The versions of the toolchain packages used in testing were
binutils-2.13.2, gcc-3.2.1 and glibc-2.3.1.
Chapter 5 - Creating the /stage1 directory
------------------------------------------
We assume you have already reached Chapter 5 of the LFS book. i.e. partition
created, filesystem made, partition mounted, $LFS/static created, lfs user
added, environment setup.
As mentioned in the introduction, we will use "/stage1" instead of "/static"
due to the fact we will be building a dynamic (shared) Chapter 5. This naming is
only a cosmetic thing. Leaving it as /static is likely to cause confusion.
Please revisit the "Creating the $LFS/static directory" chapter of the book but
instead run the following:-
mkdir $LFS/stage1
Also, don't forget to do the chown bit:-
chown lfs $LFS/stage1
The next step is to create a "/stage1" directory on the host system. Then we
"mount --bind" the $LFS/stage1 directory onto the /stage1 directory we just
created on the host:-
mkdir /stage1
mount --bind ${LFS}/stage1 /stage1
This ensures our toolchain will look in the same place (i.e. /stage1) in both
Chapters 5 and 6 (when we are inside the chroot). This is an important concept
to grasp. This aspect will become clearer as we progress.
Chapter 5 - Setting up the environment
--------------------------------------
Some adjustments to the environment setup are needed. Firstly, we do not use
the:- export CC="gcc -s" thing. We are trying to do things correctly here
and strictly speaking, the "-s" is a linker flag. Therefore we will put "-s" in
the LDFLAGS where it belongs like so:-
export LDFLAGS="-s"
Secondly, we need to add "/static/bin" to the lfs user's `PATH' like so:-
export PATH=/stage1/bin PATH
As we progressively install Chapter 5 packages into "/stage1", the tools we just
built will get picked up in the `PATH' and thus get integrated into the build
process as we move along.
Thirdly, we switch off the "hash" function of the bash shell like so:-
set +h
Bash has a useful feature whereby it uses a hash table to remember the full
pathnames of executable files to avoid multiple `PATH' searches. This
unfortunately conflicts with our intentions in that we want the newly built
tools to always come first in the `PATH'. The performance impact of disabling
this feature is negligible. To confirm the feature is switched off, simply
type:- hash at the shell prompt and it should report something like "su:
hash: hashing disabled"
One thing you'll notice throughout the build is some careful adjustment of the
CFLAGS environment variable. This is NOT TO FIDDLE WITH OPTIMISATION! The sole
purpose is to strip out the "-g" flag and thus save space during the builds. We
also throw in "-pipe" for marginally quicker builds. We do not export the CFLAGS
into the lfs user's environemnt globally because not all packages are
co-operative with their treatment of CFLAGS.
Chapter 5 - Installing binutils - Pass 1 (static)
-------------------------------------------------
This is basically the same as the current Chapter 5 binutils installation but
with an extra twist.
mkdir ../binutils-build &&
cd ../binutils-build &&
CFLAGS="-O2 -pipe" ../binutils-2*/configure --prefix=/stage1 \
--disable-nls &&
make LDFLAGS="-all-static -s" &&
make install
IMPORTANT! - Do NOT remove the binutils-build dir as we are going to reuse it a
bit further down the track. This will be explained further on, but for now, just
run the following:-
cd ld &&
make clean &&
make LDFLAGS="-all-static -s" LIB_PATH=/stage1/lib
DO NOT RUN "make install" AT THIS POINT! We are saving this for later. All will
become clear in due course.
One of the main differences is we use a prefix of /stage1 instead of the usual
$LFS/static. Note the lack of $LFS. It is worth mentioning here that having the
new binutils available from the outset is important because both glibc and gcc
perform various feature tests on the linker and assembler to determine which
software features to enable.
Chapter 5 - Installing gcc - Pass 1 (static)
--------------------------------------------
(NOTE - SHORTCUT ALERT! - if you are building from a host with a minimum gcc-3.2
already installed, it is possible to skip this gcc step. The reason the shortcut
is possible is that the next package, glibc, requires a minimum gcc-3.2. We
recommend against skipping this step because it will mean our Chapter 5 glibc
will have then been compiled with a potentially unclean gcc from the host.)
mkdir ../gcc-build &&
cd ../gcc-build &&
CFLAGS="-O2 -pipe" ../gcc-3*/configure --prefix=/stage1 \
--with-local-prefix=/stage1 --enable-languages=c \
--disable-nls --disable-shared &&
make BOOT_LDFLAGS="-static -s" BOOT_CFLAGS="-O2 -pipe" \
STAGE1_CFLAGS="-pipe" bootstrap &&
make install-no-fixedincludes &&
ln -s gcc /stage1/bin/cc
Here we start to experience some of the magic of the GNU toolchain. It is
important to realise that gcc does not search the `PATH' during it's configure
run to search for which binutils to use. It actually searches
$prefix/TARGET-TRIPLE/bin where TARGET-TRIPLE is for e.g. i686-pc-linux-gnu.
This means everything "just works" and we don't have to specify --with-ld= and
--with-as= to configure.
The extra BOOT_CFLAGS and STAGE1_CFLAGS on the make line are just extra steps to
remove "-g" from the gcc build. A few occurences still remain but we haven't yet
figured out how to weed them out.
The other interesting configure switch is --with-local-prefix=/stage1. The
purpose of this switch is to remove /usr/local/include from gcc's include search
path. It is not absolutely essential but we want to try and minimise the
influence from the host system so this is a logical thing to do. The gcc install
page[2] says:-
"The same value can be used for both --with-local-prefix and --prefix provided
it is not /usr. This can be used to avoid the default search of
/usr/local/include."
which is exactly what we want. You can see the gcc search include order by
compiling a test program with "-v" e.g. gcc -v foo.c. The following snippets of
output demonstrate the issue. This is with the "--with-local-prefix" switch:-
#include "..." search starts here:
#include <...> search starts here:
/stage1/include
/stage1/lib/gcc-lib/i686-pc-linux-gnu/3.2.1/include
/usr/include
End of search list
And this is without:-
#include "..." search starts here:
#include <...> search starts here:
/usr/local/include
/stage1/include
/stage1/lib/gcc-lib/i686-pc-linux-gnu/3.2.1/include
/usr/include
End of search list
Chapter 5 - Installing kernel headers
-------------------------------------
In normal LFS we install the kernel headers in Chapter 6. But seeing as we are
now installing a glibc in Chapter 5, we need to install the kernel headers in
Chapter 5 also.
The procedure is essentially the same as current LFS with the exception that the
headers are copied to /stage1/include. In other words, follow the procedure from
current LFS (minus the /bin/pwd hacks) and substitute "cp blah /usr/include"
with "cp blah /stage1/include".
Chapter 5 - Installing glibc
----------------------------
By this stage (assuming no shortcuts were taken) we have a new binutils and a
new gcc (both statically linked against the glibc on the host). We'll use both
of these to build the Chapter 5 glibc.
mkdir -p /stage1/etc &&
touch /stage1/etc/ld.so.conf &&
mkdir ../glibc-build &&
cd ../glibc-build &&
CFLAGS="-O2 -pipe" ../glibc-2*/configure --prefix=/stage1 \
--enable-add-ons --disable-profile \
--with-binutils=/stage1/bin \
--with-headers=/stage1/include &&
make &&
make install
Please note, the use of --with-binutils= and --with-headers= configure switches
is not strictly required. But they do ensure nothing can go wrong with regard
to the kernel headers and binutils that get used during the glibc build. If you
perform a "diff" comparison between the two configured glibc-build dirs, both
with, and without those switches, you'll surely believe it is a lot cleaner to
use them.
As mentioned above, the CFLAGS saves space during the build. If you rely solely
on the LDFLAGS from the environment then the *.a archives will still be chock
full of debug symbols. It is important to understand the glibc build system is
very sensible about how it treats the CFLAGS supplied by the user. In other
words, the CFLAGS above do not change any vital aspect of the glibc build. They
simply remove "-g" and add "-pipe".
The other important thing to realise is that glibc is pretty well self contained
in terms of it's build mechanics. No fancy linker switches are required. It
doesn't rely on compiler or linker defaults as all the required build switches
are supplied "by hand". e.g. you will see stuff like:-
-Wl,-dynamic-linker=/stage1/lib/ld-linux.so.2
during the glibc build. What is shown there is simply an example of how glibc's
build system overrides gcc's default behaviour of passing:-
-dynamic-linker=/lib/ld-linux.so.2
to the linker (ld) as it goes about it's business of constructing the binaries
and libraries.
Linker scripts and library search order - A quick overview
----------------------------------------------------------
At this juncture, it is worth discussing how ld (the linker) performs it's
magic. On a normal LFS system you will find the linker scripts located in the
/usr/lib/ldscripts dir. They have names like elf_i386.x, elf_i386.xc and so on.
There are 13 of these scripts altogether but only the 8 that start with elf*
concern us. The others are involved when creating a.out format binaries which
basically means never. The a.out format was superceded years ago by the more
modern ELF format.
The linker scripts for our new toolchain in /stage1 are located in
/stage1/TARGET-TRIPLE/lib/ldscripts. If you view the scripts, you'll notice the
default linker search path (as indicated by the SEARCH_DIR variables) is:-
$prefix/TARGET-TRIPLE/lib, $prefix/lib, /usr/local/lib, /lib, /usr/lib
where $prefix is the value of the --prefix configure option and TARGET-TRIPLE
is the cpu-arch-os triple for the host (e.g. i686-pc-linux-gnu). Anything linked
by ld using these linker scripts will link against our new glibc installed into
/stage1/lib because /stage1/lib comes before /usr/lib in the search path. So far
so good. This is exactly what we want. However, what we really need is to remove
the /usr/local/lib, /lib, and /usr/lib from the search path altogther. In this
way we can be 100% certain that nothing links against the libraries on the host
system. There are a few ways we can achieve this:-
1. edit all the linker scripts by hand
2. perform some sort of sed substitution on each of the scripts
3. utilise the LIB_PATH variable in ld's Makefile to our advantage
No 3 is the easiest solution. A small mention of LIB_PATH was made back in the
pass 1 binutils section. A bit further down, you'll see how it's usage helps us.
Gcc's specs file - A quick overview
-----------------------------------
>From the gcc info page:-
"`gcc' is a driver program. It performs its job by invoking a sequence of other
programs to do the work of compiling, assembling and linking. GCC interprets
its command-line parameters and uses these to deduce which programs it should
invoke, and which command-line options it ought to place on their command lines.
This behavior is controlled by "spec strings". In most cases there is one spec
string for each program that GCC can invoke, but a few programs have multiple
spec strings to control their behavior. The spec strings built into GCC can be
overridden by using the `-specs=' command-line switch to specify a spec file.
"Spec files" are plaintext files that are used to construct spec strings."
To find out where the specs file is located simply type in "gcc -v" like so:-
gws@tigers-lfs:~$ gcc -v 2>&1 | head -n 1
Reading specs from /static/lib/gcc-lib/i686-pc-linux-gnu/3.2.1/specs
If you examine a normal dynamic executable using the readelf utiltity, you can
determine which dynamic linker (i.e. program interpreter) the executable was
compiled for. Take a look at this example:-
gws@tigers-lfs:~/src$ readelf -a foo | grep interpreter
[Requesting program interpreter: /lib/ld-linux.so.2]
If we want to change the program interpreter that gets embedded into our
binaries we need to utilise the linker switch -dynamic-linker e.g.
-Wl,-dynamic-linker=/stage1/lib/ld-linux.so.2. But forcing this onto every
package we compile can be quite troublesome. It is much easier to simply force
the issue by adjusting the specs string in the specs file to point where we
want.
There are a few ways we can achive this:-
1. edit, sed or patch the specs file
2. patch the gcc source before we compile it
Both methods are used below, depending on which stage of the build process we
are at.
Chapter 5 - Locking in our new glibc
------------------------------------
Now that we have the lectures out of the way, it is down to business. By now we
have our Chapter 5 glibc installed and operational. This is the point in the
proceedings where we need to ensure that we do not link against the libraries on
the host system.
The first thing to do is edit our gcc specs file. Simply edit the file by hand
using your favourite editor and replace the single occurrence of
"/lib/ld-linux.so.2" with "/stage1/lib/ld-linux.so.2".
(NOTE - a patch or a sed might be more appriate above - dunno)
The next thing to do is adjust the linker scripts. Remember back in the binutils
pass 1 section we added a couple of extra commands at the end? Well here is
where we utilise those commands. Simply cd back into your binutils-build dir and
run the following:-
cd ld &&
make install-data-local
This installs the adjusted linker scripts. The linker scripts now contain no
mention of /lib, /usr/lib or /usr/local/lib. From this point onwards, everything
will link only against the libs in /stage1/lib.
Chapter 5 - Installing binutils - Pass 2 (shared)
-------------------------------------------------
Here we reinstall binutils. It will be dynamically linked against the new glibc
(guaranteed due to the "lock in" steps we took in the last section).
mkdir ../binutils-build &&
cd ../binutils-build &&
CFLAGS="-O2 -pipe" ../binutils-2*/configure --prefix=/stage1 \
--disable-nls &&
make -e LDFLAGS="-s" LIB_PATH=/stage1/lib &&
make install
The only curious command here is the use of "make -e". We need to do this so
that the LIB_PATH=/stage1/lib gets passed down to the sub-make process during
the build of ld. Some people may consider this "impure" or a bit of a security
risk. The use of "make -e" gives variables taken from the environment precedence
over variables from makefiles. Another solution would be to perform a sed on the
ld/Makefile before compilation. Or we could even employ a similar (cd ld; make
clean; make LIB_PATH=/static/lib; make install-data-local) technique as per our
pass 1.
Chapter 5 - Installing gcc - Pass 2 (shared)
--------------------------------------------
(NOTE - SHORTCUT ALERT! - this step could be considered optional as we already
have a new gcc to begin compiling Chapter 6 with. However that gcc is statically
linked against the host's glibc. By recompiling gcc now, we'll end up with a
completely shared toolchain linked against the new glibc. And it will be very
clean indeed.)
Before we start this recompile of gcc, we really need to ensure that the
traditional spec string for the dynamic linker doesn't somehow influence the
build. We therefore patch gcc (the file is gcc/config/i386/linux.h). It is just
a simple substition of "/lib/ld-linux.so.2" with "/stage1/lib/ld-linux.so.2". Or
again, we could perform a sed instead. It doesn't really matter. The end result
will be the same as manually editing the specs file at the end of the build like
we did earlier on.
patch -Np1 -i ../gcc-3.2.1.specs.patch &&
mkdir ../gcc-build &&
cd ../gcc-build &&
CFLAGS="-O2 -pipe" ../gcc-3*/configure --prefix=/stage1 \
--with-local-prefix=/stage1 --enable-languages=c --disable-nls \
--enable-shared &&
make BOOT_LDFLAGS="-s" BOOT_CFLAGS="-O2 -pipe" \
STAGE1_CFLAGS="-pipe" bootstrap &&
make install-no-fixedincludes
The main difference between here and pass 1 is the --enable-shared configure
switch to build the shared gcc libs.
Chapter 5 - Toolchain Summary
-----------------------------
At this stage, we have compiled gcc twice, binutils twice and glibc once. A bit
excessive you think? Not quite, when you realise how clean the toolchain really
is. That feeling of cleanliness is very powerful and worth every bit of effort
in our opinion.
Chapter 5 - Remaining packages - (shared)
-----------------------------------------
The list of remaining packages in Chapter 5 is still open to debate. We
definitely need to add ncurses. Additionally, the build order is still under
review to maximise the cleanliness effect.
TODO
====
- remainder of Chapter 5 packages
- Chapter 6
- look at "make check" stuff
- look at ways to remove /usr/include from gcc's include search path
- look at ways to remove unwanted dirs from "gcc -print-search-dirs"
- look at gcc-2.95.x for kernel compiles
- look at using gcc's -enable-version-specific-runtime-libs for installing
multilple versions of gcc into the one prefix
- look at NLS stuff
- look at C++ stuff
CREDITS
=======
Ryan - Devised the whole scheme. Bucket loads of testing and research. Helped
with documentation.
Greg - Wrote the bulk of the documentation. Helped refine the process, Helped
with testing and research. Initially ranted on lfs-dev about potential
build flaws and thus spurred Ryan into action :-)
lfs-dev folk - input and ideas.
FOOTNOTES
=========
1. http://www.linuxbase.org/impl/
2. http://gcc.gnu.org/install/configure.html |
|
IP卡
狗仔卡
提升卡
置顶卡
沉默卡
喧嚣卡
变色卡
显身卡
楼主