.. a major configuration cleanup
... major enhancement of automake support.
... and it contains a *major* breakthough:
Automake support for libchip and libmisc *LEAF* directories.
To implement this I have used several nasty tricks
* The basical trick is to wrap an old Makefile.in's contents into a
Makefile.am and still continue to use (i.e include) the old
*.cfg files.
* Replaced each INSTALL_IF_CHANGE and INSTALL_VARIANT with make
dependencies
* Add a gnu-make ifdef AUTOMAKE to main.cfg to avoid conflicts between
automake and RTEMS make rules
* Replaced each install:: and preinstall:: rule with make dependencies
* Replaced SUB_DIRS with SUBDIRS in all Makefile.ins (Automake
convention)
* Removed each manually added autoconf substitution which automake
performs automatically.
This is not yet full automake support, because using the temporary
installation directory, preinstallation in general and building variants
are in contradiction to automake's basic working principles ...
... the new Makefile.ams work still somewhat clumsy
... nevertheless they work (quite well).
WARNING:
At first glance this patch is small, but
* it affects the whole configuration system.
* it opens the road to introducing automake to all Makefile.ins
currently not being under automake control.
JOEL> Does this remove or add any files?
Both, all Makefile.ins below libchip and libmisc get replaced with
Makefile.ams.
unnecessarily uses any variables defined by the BSP. On this
sweep, use of BSP_Configuration and Cpu_table was eliminated.
A significant part of this modification was the addition of
macros to access fields in the RTEMS configuration structures.
This is necessary to strengthen the division between the BSP independent
parts of RTEMS and the BSPs themselves. This started after
comments and analysis by Ralf Corsepius <corsepiu@faw.uni-ulm.de>.
particular, using bsp.h, or getting information from the BSP which
should properly be obtained from RTEMS is forbidden. This is
necessary to strengthen the division between the BSP independent
parts of RTEMS and the BSPs themselves. This started after
comments and analysis by Ralf Corsepius <corsepiu@faw.uni-ulm.de>.
The changes primarily eliminated the need to include bsp.h and
peeking at BSP_Configuration. The use of Cpu_table in each
BSP needs to be eliminated.
to move RTEMS more to automake/autoconf and GNU compliance.
Finally, here they are: the "big-patch" patches - merged into one big
patch (~1.5MB).
Sorry for the delay, but testing took much more time than I had expected
- esp. reworking the acpolish script triggered many more tiny issues
than I had expected (cf. below).
At least, now you've got something to spend your weekend with :-.
WARNINGS:
* I've gone a little (??) further than I had announced before.
* Several directories have been moved.
* Several files have been added and removed
* I have tested it with many BSPs/CPUs and a variety of permutiations of
configuration flags, but not with all.
* Most parts of the patch are automatically generated, however there are
many tiny manual modifications.
APPLYING THE PATCH:
./autogen -c
mkdir tools
mv c/src/exec/score/tools tools/cpu
mv c/build-tools tools/build
mv c/update-tools tools/update
patch -p1 -E < rtems-rc-19990709-0.diff
./autogen
If the patch doesn't apply to rtems-cvs, I would suggest that you should
try to apply it brute-force and then to run tools/update/rtems-polish.sh
-ac -am afterwards. A recursive diff between rtems-19990709 + patch and
rtems-cvs + patch then should report only a few dozen significant
changes to configuration files which need to be merged manually (IIRC, I
did not change any source files).
*** Attention: There are files to be removed, moved, copied and added
in/to CVS!
NEWS/CHANGES:
1. Configuration takes place in 3 stages: 1. per host (toplevel
configure script), 2. per target (c/configure), 3. per bsp
c/src/configure automatically triggered from ./configure and
c/Makefile.am.
2. Building of subdirectory c/ takes place in c/$(target_alias) for
cross-targets in c/ for native targets
3. Building of subdirectory c/src takes place in c/${target_alias}/<bsp>
for cross-targets, c/<bsp> for native targets
4. c/build-tools moved to tools/build
5. c/src/exec/score/cpu/tools moved to tools/cpu (=cpu-tools split out)
6. c/update-tools moved to tools/update
7. New subdirectory c/src/make, handles files from make/ on a per BSP
basis
8. Maintainer mode support: Ie. if configuring with
--enable-maintainer-mode disabled (the default), then tracking of many
dependencies will be disabled in Makefiles. Esp. many dependencies for
auto* generated files will be switched off in Makefiles. Ie. if not
using "--enable-maintainer-mode" many auto* generated files will not be
updated automatically, i.e. normal users should not be required to have
auto* tools anymore (untested).
9. Independent configuration scripts for / (toplevel), tools/build,
tools/cpu, tools/update, c/, c/src/, c/src/exec, c/src/lib, c/src/tests,
c/src/make
10. Automake support for all directories above and besides c/src
11. "preinstall" now is implemented as depth-first recursive make target
12. host compiled tools (exception bsp-tools) are accessed in location
in the build tree instead of inside the build-tree when building RTEMS.
13. RTEMS_ROOT and PROJECT_ROOT now point to directories inside the
build-tree - many tiny changes as consequence from this.
14. --with-cross-host support removed (offically announced obsolete by
cygnus)
15. Changing the order of building libraries below c/src/lib/
16. Former toplevel configure script broken into aclocal/*.m4 macros
17. Newlib now detected by configure macros, RTEMS_HAS_NEWLIB removed
from *cfg.
18. sptables.h now generated by autoconf
19. Rules for "mkinstalldirs temporary installation tree" moved from
c/Makefile to subdirectories.
20. Cpu-tools do not get installed.
21. FIX: Use ACLOCAL_AMFLAGS instead of ACLOCAL = -I ... in Makefile.ams
which are in directories with own configure scripts.
22. Hardcoding BSP names into libbsp/.../tools to avoid RTEMS_BSP get
overridden from the environment.
22. FIX: Handling of MP_PIECES in various Makefiles
23. FIX: Removing "::" rules from some Makefile.ins
24. FIX: File permission chaos: (-m 444 and -m 555 vs. -m 644 and -m
755) - Now all include files use -m 644.
25. Removed many gnumake-conditionals in Makefile.ins - Partially
replaced with automake-conditional, partially replaced with
conditionalized Makefile variables (... _yes_V)
26. Massively reworked acpolish: acpolish now parses Makefile.ins and
interprets parts of the Makefile.ins.
27. FIX: Some $(wildcard $(srcdir)/*.h) macros removed / replaced with
explicit lists of files in Makefile.ins.
28. FIX: Replacing MKLIB with RANLIB in Makefile.ins
29. HACK: Add preinstallation for pc386 specific
$(PROJECT_RELEASE)/BootImgs directory
... many more details, I can't recall
KNOWN BUGS:
1. make [debug|profile]_install do not do what they are promissing.
"make [debug|profile] install" does what "make [debug|profile]_install"
has been doing. Proposal: remove [debug|profile]_install
2. Dependencies between temporary installation tree and source tree are
not yet handled correctly.
3. Dependencies between temporary installation tree and source tree are
handled ineffencently (Using INSTALL_CHANGE instead of make
dependencies)
4. RTEMS_ROOT, PROJECT_ROOT, top_builddir, RTEMS_TOPdir now are
redundant.
5. The new configure scripts still are in their infancy. They contain
redundant checks and might still contain bugs, too.
6. RTEMS autoconf Makefile.ins use a mixture of configuration
information gathered in c/$(target_alias)/<bsp>/make and of information
collected from their configure scripts.
7. make dist is not fully functional
8. Subdirectory host-/build-/target- configure options (--target,
--host, --build) do not conform to Cygnus/GNU conventions.
9. Some RTEMS autoconf Makefile.in's makefile targets are not supported
in automake Makefile.ams/ins (e.g. get, clobber).
10. Some automake standard targets are not propagated from toplevel and
c/Makefile.am to autoconf subdirectories (eg. make dist).
11. rpcgen generated files are not part of the source-tree (Automake
conventions favor supplying generated files inside the source-tree,
however there is no support for rpcgen generated files in automake, cf.
yacc/lex support in automake).
12. RTEMS_HAS_RDBG handling is flaky. make/*.cfg use RTEMS_HAS_RDBG per
CPU, while librdb's sources can only be built per BSP. Raises the more
general question whether librdbg located correctly in the source-tree.
13. All make/*cfg files are configured per cpu, currently there is no
location to store per-bsp configuration information --> bsp.cfg, per
aconfig.h?
14. "make install" without having run "make all" beforehand does not
work.
15. handling of --enable-multiprocessing seems to be broken in
make/custom/*
16. Makefile.ins still exploit many gmake features.
17. File permisson chaos on libraries (no explict -m for
libraries/rels/etc).
18. mcp750 Makefiles are broken (Note: I *do* mean buggy - I am not
talking about "not-conforming to conventions", here :-).
19. Dependencies between configure scripts are not handled, eg. aborting
"make RTEMS_BSP=<bsp>" can leave the build-tree in an unusable state.
20. "make clean" does not delete <build-tree>/<bsp>. This is intentional
for now, because rerunning "make" after "make clean" requires an
explicit "make preinstall" afterwards now. This should be done
automatically, but doesn't work in this case for now. To work around
this problem <build-tree>/<bsp> is kept during "make clean" for now
(HACK).
TODO:
1. split out host-compiled bsp-tools
2. Use Cygnus/GNU standards for cross-compiling target-subdir
(CC=CC_FOR_TARGET .. configure --host=${target_alias}
--build=`config.guess'}), to be added to toplevel configure script after
splitting out bsp-tools.
3. Exploit per cpu support directory (c/src/<cpu>)- Splitting out
per-cpu libraries - Are there any?
4. Further automake support
5. Converting subdirectories into standalone / self-contained
subdirectories (Esp. moving their headers to the same common root as
their sources, eg. mv lib/include/rtems++
lib/librtems++/include/rtems++) - This is the main obstacle which
prevents moving further towards automake.
6. Propagating values from *.cfg into Makefiles instead of propagating
them at make time via Makefile-fragments (i.e. try to avoid using
*.cfg).
7. Testing on cygwin host (I *do* expect cygwin specific problems).
8. The ARCH in o-$(ARCH)-$(VARIANT) build-subdirectories is not needed
anymore.
GENERAL ISSUES:
1. Temporary installation tree -- Ian and I seem to disagree basically.
Though I think that I understand his argumentation, I do not share it.
IMO, his way of using the buildtree is mis-using the build-tree, relying
on an inofficial feature of RTEMS's current implementation, which
doesn't even work correctly in the current build-tree, though it
attempts hard to do so. From my very POV, it unnecessarily complicates
the structures of the source- and build-trees. It is not supported by
automake (No automatic generation for the necessary rules) and
complicates the transition to automake significantly (Generating the
rules with an enhanced version of acpolish could be possible).
As Ian correctly pointed out, here a management decision is needed -
though I don't see the need to draw this decision in short terms.
2. preinstallation generally is a sure means to spoil the structure of
the source tree, IMHO (No ranting intended, I am completly serious about
this one). eg. through tree dependencies. The worst problem related to
this I have found in the meantime is bsp_specs. bsp_specs is part of
libbsp, ie. there is *no* way to build *any* part of the source tree
*without* having a BSP *preinstalled*.
Note: This issue is related to issue 1., but is not identical - The
difference is the change of the order make rules have to be triggered.
While preinstallation triggers rules spread all over the source tree
before a "make all" can be run, a temporary installation tree could also
be installed by post "make all" hooks (all-local:, to be run after make
all in a directory is completed) if the directories' dependencies would
be a tree,
3. Stuctural dependencies between subdirectories.
4. Depth of the source tree (Prevents multilibbing and introduces many
unnecessary configure scripts).
5. per cpu vs. per bsp configuration (There are no real per-cpu parts
yets :-).
6. automake does not support $makefiles in AC_OUTPUT. Unlike before, we
now should try to avoid RTEMS_CHECK_MAKEFILE and to hard-code as much
paths to Makefiles as possible.
7. General redesign of the source tree
8. Main installation point - Changing it to ${prefix}/${target_alias}. ?
Besides item 8. (which is a must, IMHO), as far as I see most of them
can not be solved soon and will remain issues in the mid- to long-term
:-.
REMARKS:
* You (as the maintainer) should always use --enable-maintainer-mode
when building RTEMS to ensure that maintainer mode generated files (esp.
those in c/src/make) will be updated when make/* files have changed.
* Use @RTEMS_BSP@ in Makefile.ins and Makefile.ams below c/src/,
$(RTEMS_BSP) or ${RTEMS_BSP} will be overridden from environment
variables when using make RTEMS_BSP="....".
* c/src/make is a temporary cludge until configuration issues are
solved. At the moment it is configured per bsp, but contains
per-target/cpu info only. Its main purpose now is to circumvent
modifying make/*.cfg files, because I consider make/* to be frozen for
backward compatibilty.
* This patch should only affect configuration files. At least I do not
remember having touched any source files.
* To build the bare bsp you now need to mention it in --enable-rtemsbsp.
Example: building gensh1 and sh1/bare simultaneously:
../rtems-rc-19990709-1/configure --target=sh-rtems \
--enable-rtemsbsp="bare gensh1" \
--prefix=/tmp/rtems \
--enable-bare-cpu-cflags='-DMHZ=20 -m1
-DCPU_CONSOLE_DEVNAME=\"/dev/null\"' \
--enable-bare-cpu-model=sh7032 \
--enable-maintainer-mode \
--enable-cxx
make
make install
* The next steps in development would be to split out bsp-tools and then
to change to Cygnus/GNU canonicalization conventions for building the c/
subdirectory afterwards (i.e. many standard AC_*.m4 macros could be used
instead of customized versions)
FINAL REMARK:
The issues mentioned in the lists above sound much worser than the
situation actually is. Most of them are not specific to this patch, but
are also valid for the snapshot. I just wrote down what I came across
when working on the patch over the last few weeks.
I wouldn't be too surprised if you don't like the patch at the current
point in development. I am willing to discuss details and problems, I
also have no problem if you would post-pone applying this patch to times
after 4.1, but rejecting it as a whole for all times would be a false
management decision, IMHO.
Therefore I would suggest that you, if your time constaints allow it,
should at least play a little while with this patch to understand what
is going on and before drawing a decision on how to handle this
proposal. I know this patch is neither perfect nor complete, but I
consider it to be a major breakthrough. Don't be anxious because of the
size of the patch, the core of the patch is rather small, the size is
mainly the side effect of some systematic cleanups inside the Makefiles
(result of acpolish).
Feel free to ask if you encounter problems, if you don't understand
something or if you meet bugs - I am far from being perfect and am
prepared to answer them.
Ralf.
--
Ralf Corsepius
Forschungsinstitut fuer Anwendungsorientierte Wissensverarbeitung (FAW)
Helmholtzstr. 16, 89081 Ulm, Germany Tel: +49/731/501-8690
mailto:corsepiu@faw.uni-ulm.de FAX: +49/731/501-999
http://www.faw.uni-ulm.de
First, the unlimited patch. I have compiled the unlmited patch for the
Linux posix BSP only and it seems to work cleanly. I would like a really
major application run on this change before commiting as the changes are
very core and significant. I am currently building all the tests to run.
I have no targets suitable to test on at the moment.
I have tested the patch for inline functions and macros.
Turning macros on has found some core bugs. I have fixed these but have
not run all the tests. Please review the patch for these changes. They
are:
1) The conditional compilation for MP support broke the core messages
code. You cannot embed a conditional macro in another macro. The Send
and Urgent Send calls are macros.
2) User extensions handler initialisation now has two parameters. I have
updated the macros to support the extra parameter.
The patch also contains the gcc-target-default.cfg fix required to build
the kernel. More of a by product than a fix for you.
+ POSIX threads
+ Ada tasks
+ POSIX threads API support
+ POSIX condition variables
+ POSIX keys (still do not account for data fields)
+ POSIX mutexes
+ POSIX queued signals
+ added macro for extra memory for task stacks
+ default value for CONFIGURE_MAXIMUM_FAKE_ADA_TASKS
+ extra memory for Ada task stacks
code in each BSP's bspstart.c. These changes were:
+ confdefs.h now knows libio's semaphore requirements
+ shared/main.c now copies Configuration to BSP_Configuration
+ shared/main.c fills in the Cpu_table with default values
This removed the need for rtems_libio_config() and the constant
BSP_LIBIO_MAX_FDS in every BSP. Plus now the maximum number of open
files can now be set on the gcc command line.
to multiple. This lets the stack check extension be installed
at system initialization time and avoids the BSP having to
even know about its existence.
for GNAT. It was the number of Ada tasks when in fact the run-time
only required a single key.
Also added the CONFIGURE_MAXIMUM_FAKE_ADA_TASKS constant to account
for resources allocated for each non-Ada task/thread which invokes
the Ada run-time implicitly through an Ada call.
GNU autoconf. This is the first large step in allowing an RTEMS
user to perform a one-tree build (per crossgcc FAQ) including RTEMS
in the build process. With this change RTEMS is configured in
built in the same style as the GNU tools, yet retains the basic
structure of its traditional Makefiles (ala Tony Bennett).
Jiri Gaisler (jgais@wd.estec.esa.nl) deserves (and received)
a big thank you for doing this.
There are still issues to be resolved but as of this commit, all target
which can be built on a linux host have been using a modified version
of the source Jiri submitted. This source was merged and most targets
built in the tree before this commit.
There are some issues which remain to be resolved but they are primarily
related to host OS dependencies, script issues, the use of gawk
for hack_specs, and the dependence on gcc snapshots. These will
be resolved.
