This chapter describes how the runnable Emacs executable is dumped with the preloaded Lisp libraries in it, how storage is allocated, and some internal aspects of GNU Emacs that may be of interest to C programmers.
This section explains the steps involved in building the Emacs executable. You don't have to know this material to build and install Emacs, since the makefiles do all these things automatically. This information is pertinent to Emacs maintenance.
Compilation of the C source files in the `src' directory produces an executable file called `temacs', also called a bare impure Emacs. It contains the Emacs Lisp interpreter and I/O routines, but not the editing commands.
The command `temacs -l loadup' uses `temacs' to create the real runnable Emacs executable. These arguments direct `temacs' to evaluate the Lisp files specified in the file `loadup.el'. These files set up the normal Emacs editing environment, resulting in an Emacs that is still impure but no longer bare.
It takes a substantial time to load the standard Lisp files. Luckily, you don't have to do this each time you run Emacs; `temacs' can dump out an executable program called `emacs' that has these files preloaded. `emacs' starts more quickly because it does not need to load the files. This is the Emacs executable that is normally installed.
To create `emacs', use the command `temacs -batch -l loadup dump'. The purpose of `-batch' here is to prevent `temacs' from trying to initialize any of its data on the terminal; this ensures that the tables of terminal information are empty in the dumped Emacs. The argument `dump' tells `loadup.el' to dump a new executable named `emacs'.
Some operating systems don't support dumping. On those systems, you must start Emacs with the `temacs -l loadup' command each time you use it. This takes a substantial time, but since you need to start Emacs once a day at most--or once a week if you never log out--the extra time is not too severe a problem.
You can specify additional
files to preload by writing a library named `site-load.el'
that loads them. You may need to increase the value of
PURESIZE, in `src/puresize.h', to make room
for the additional data. (Try adding increments of 20000 until it
is big enough.) However, the advantage of preloading additional
files decreases as machines get faster. On modern machines, it is
usually not advisable.
After `loadup.el' reads `site-load.el', it
finds the documentation strings for primitive and preloaded
functions (and variables) in the file `etc/DOC' where they
are stored, by calling Snarf-documentation (see
section Access to Documentation
Strings).
You can specify other Lisp expressions to execute just before dumping by putting them in a library named `site-init.el'. This file is executed after the documentation strings are found.
If you want to preload function or variable definitions, there are three ways you can do this and make their documentation strings accessible when you subsequently run Emacs:
nil value for
byte-compile-dynamic-docstrings as a local variable in
each these files, and load them with either `site-load.el'
or `site-init.el'. (This method has the drawback that the
documentation strings take up space in Emacs all the time.)
It is not advisable to put anything in `site-load.el' or `site-init.el' that would alter any of the features that users expect in an ordinary unmodified Emacs. If you feel you must override normal features for your site, do it with `default.el', so that users can override your changes if they wish. See section Summary: Sequence of Actions at Start Up.
If you want to use this function in an Emacs that was already dumped, you must run Emacs with `-batch'.
Emacs Lisp uses two kinds of storage for user-created Lisp objects: normal storage and pure storage. Normal storage is where all the new data created during an Emacs session is kept; see the following section for information on normal storage. Pure storage is used for certain data in the preloaded standard Lisp files--data that should never change during actual use of Emacs.
Pure storage is allocated only while `temacs' is
loading the standard preloaded Lisp libraries. In the file
`emacs', it is marked as read-only (on operating systems
that permit this), so that the memory space can be shared by all
the Emacs jobs running on the machine at once. Pure storage is not
expandable; a fixed amount is allocated when Emacs is compiled, and
if that is not sufficient for the preloaded libraries,
`temacs' crashes. If that happens, you must increase the
compilation parameter PURESIZE in the file
`src/puresize.h'. This normally won't happen unless you
try to preload additional libraries or add features to the standard
ones.
This function is a no-op except while Emacs is being built and dumped; it is usually called only in the file `emacs/lisp/loaddefs.el', but a few packages call it just in case you decide to preload them.
defun should make a copy of the function
definition in pure storage. If it is non-nil, then the
function definition is copied into pure storage. This flag is t while loading all of the basic
functions for building Emacs initially (allowing those functions to
be sharable and non-collectible). Dumping Emacs as an executable
always writes nil in this variable, regardless of the
value it actually has before and after dumping.
You should not change this flag in a running Emacs.
When a program creates a list or the user defines a new function (such as by loading a library), that data is placed in normal storage. If normal storage runs low, then Emacs asks the operating system to allocate more memory in blocks of 1k bytes. Each block is used for one type of Lisp object, so symbols, cons cells, markers, etc., are segregated in distinct blocks in memory. (Vectors, long strings, buffers and certain other editing types, which are fairly large, are allocated in individual blocks, one per object, while small strings are packed into blocks of 8k bytes.)
It is quite common to use some storage for a while, then release it by (for example) killing a buffer or deleting the last pointer to an object. Emacs provides a garbage collector to reclaim this abandoned storage. (This name is traditional, but "garbage recycler" might be a more intuitive metaphor for this facility.)
The garbage collector operates by finding and marking all Lisp objects that are still accessible to Lisp programs. To begin with, it assumes all the symbols, their values and associated function definitions, and any data presently on the stack, are accessible. Any objects that can be reached indirectly through other accessible objects are also accessible.
When marking is finished, all objects still unmarked are garbage. No matter what the Lisp program or the user does, it is impossible to refer to them, since there is no longer a way to reach them. Their space might as well be reused, since no one will miss them. The second ("sweep") phase of the garbage collector arranges to reuse them.
The sweep phase puts unused
cons cells onto a free list for future allocation;
likewise for symbols and markers. It compacts the accessible
strings so they occupy fewer 8k blocks; then it frees the other 8k
blocks. Vectors, buffers, windows, and other large objects are
individually allocated and freed using malloc and
free.
Common Lisp note: Unlike other Lisps, GNU Emacs Lisp does not call the garbage collector when the free list is empty. Instead, it simply requests the operating system to allocate more storage, and processing continues until
gc-cons-thresholdbytes have been used.This means that you can make sure that the garbage collector will not run during a certain portion of a Lisp program by calling the garbage collector explicitly just before it (provided that portion of the program does not use so much space as to force a second garbage collection).
gc-cons-threshold bytes of Lisp data since the
previous garbage collection.) garbage-collect returns a list containing the
following information:
((used-conses . free-conses) (used-syms . free-syms) (used-miscs . free-miscs) used-string-chars used-vector-slots (used-floats . free-floats) (used-intervals . free-intervals))
Here is an example:
(garbage-collect)
=> ((106886 . 13184) (9769 . 0)
(7731 . 4651) 347543 121628
(31 . 94) (1273 . 168))
Here is a table explaining each element:
nil, Emacs displays a message at the beginning and
end of garbage collection. The default value is nil,
meaning there are no such messages.
The initial threshold value is 400,000. If you specify a larger value, garbage collection will happen less often. This reduces the amount of time spent garbage collecting, but increases total memory use. You may want to do this when running a program that creates lots of Lisp data.
You can make collections more frequent by specifying a smaller
value, down to 10,000. A value less than 10,000 will remain in
effect only until the subsequent garbage collection, at which time
garbage-collect will set the threshold back to
10,000.
The value return by garbage-collect describes the
amount of memory used by Lisp data, broken down by data type. By
contrast, the function memory-limit provides
information on the total amount of memory Emacs is currently
using.
You can use this to get a general idea of how your actions affect the memory usage.
These functions and variables give information about the total
amount of memory allocation that Emacs has done, broken down by
data type. Note the difference between these and the values
returned by (garbage-collect); those count objects
that currently exist, but these count the number or size of all
allocations, including those for objects that have since been
freed.
Lisp primitives are Lisp functions implemented in C. The details of interfacing the C function so that Lisp can call it are handled by a few C macros. The only way to really understand how to write new C code is to read the source, but we can explain some things here.
An example of a special form is the definition of
or, from `eval.c'. (An ordinary function
would have the same general appearance.)
DEFUN ("or", For, Sor, 0, UNEVALLED, 0,
"Eval args until one of them yields non-nil; return that value.\n\
The remaining args are not evalled at all.\n\
If all args return nil, return nil.")
(args)
Lisp_Object args;
{
register Lisp_Object val;
Lisp_Object args_left;
struct gcpro gcpro1;
if (NULL (args))
return Qnil;
args_left = args;
GCPRO1 (args_left);
do
{
val = Feval (Fcar (args_left));
if (!NULL (val))
break;
args_left = Fcdr (args_left);
}
while (!NULL (args_left));
UNGCPRO;
return val;
}
Let's start with a precise explanation of the arguments to the
DEFUN macro. Here is a template for them:
DEFUN (lname, fname, sname, min, max, interactive, doc)
or.
For.
Remember that the arguments must be of type
Lisp_Object; various macros and functions for creating
values of type Lisp_Object are declared in the file
`lisp.h'.
or allows a minimum of zero
arguments.
UNEVALLED, indicating a special form that receives
unevaluated arguments, or MANY, indicating an
unlimited number of evaluated arguments (the equivalent of
&rest). Both UNEVALLED and
MANY are macros. If max is a number, it may
not be less than min and it may not be greater than
seven.
interactive in a Lisp
function. In the case of or, it is 0 (a null pointer),
indicating that or cannot be called interactively. A
value of "" indicates a function that should receive
no arguments when called interactively.
After the call to the DEFUN macro, you must write
the argument name list that every C function must have, followed by
ordinary C declarations for the arguments. For a function with a
fixed maximum number of arguments, declare a C argument for each
Lisp argument, and give them all type Lisp_Object.
When a Lisp function has no upper limit on the number of arguments,
its implementation in C actually receives exactly two arguments:
the first is the number of Lisp arguments, and the second is the
address of a block containing their values. They have types
int and Lisp_Object *.
Within the function For itself, note the use of the
macros GCPRO1 and UNGCPRO.
GCPRO1 is used to "protect" a variable from garbage
collection--to inform the garbage collector that it must look in
that variable and regard its contents as an accessible object. This
is necessary whenever you call Feval or anything that
can directly or indirectly call Feval. At such a time,
any Lisp object that you intend to refer to again must be protected
somehow. UNGCPRO cancels the protection of the
variables that are protected in the current function. It is
necessary to do this explicitly.
For most data types, it suffices to protect at least one pointer to the object; as long as the object is not recycled, all pointers to it remain valid. This is not so for strings, because the garbage collector can move them. When the garbage collector moves a string, it relocates all the pointers it knows about; any other pointers become invalid. Therefore, you must protect all pointers to strings across any point where garbage collection may be possible.
The macro GCPRO1 protects just one local variable.
If you want to protect two, use GCPRO2 instead;
repeating GCPRO1 will not work. Macros
GCPRO3 and GCPRO4 also exist.
These macros implicitly use local variables such as
gcpro1; you must declare these explicitly, with type
struct gcpro. Thus, if you use GCPRO2,
you must declare gcpro1 and gcpro2. Alas,
we can't explain all the tricky details here.
You must not use C initializers for static or global variables unless they are never written once Emacs is dumped. These variables with initializers are allocated in an area of memory that becomes read-only (on certain operating systems) as a result of dumping Emacs. See section Pure Storage.
Do not use static variables within functions--place all static
variables at top level in the file. This is necessary because Emacs
on some operating systems defines the keyword static
as a null macro. (This definition is used because those systems put
all variables declared static in a place that becomes read-only
after dumping, whether they have initializers or not.)
Defining the C function is not enough to make a Lisp primitive available; you must also create the Lisp symbol for the primitive and store a suitable subr object in its function cell. The code looks like this:
defsubr (&subr-structure-name);
Here subr-structure-name is the name you used as the
third argument to DEFUN.
If you add a new primitive to a file that already has Lisp
primitives defined in it, find the function (near the end of the
file) named syms_of_something, and add the
call to defsubr there. If the file doesn't have this
function, or if you create a new file, add to it a
syms_of_filename (e.g.,
syms_of_myfile). Then find the spot in
`emacs.c' where all of these functions are called, and add
a call to syms_of_filename there.
The function syms_of_filename is also
the place to define any C variables that are to be visible as Lisp
variables. DEFVAR_LISP makes a C variable of type
Lisp_Object visible in Lisp. DEFVAR_INT
makes a C variable of type int visible in Lisp with a
value that is always an integer. DEFVAR_BOOL makes a C
variable of type int visible in Lisp with a value that
is either t or nil.
If you define a file-scope C variable of type
Lisp_Object, you must protect it for
garbage-collection by calling staticpro in
syms_of_filename, like this:
staticpro (&variable);
Here is another example function, with more complicated arguments. This comes from the code in `window.c', and it demonstrates the use of macros and functions to manipulate Lisp objects.
DEFUN ("coordinates-in-window-p", Fcoordinates_in_window_p,
Scoordinates_in_window_p, 2, 2,
"xSpecify coordinate pair: \nXExpression which evals to window: ",
"Return non-nil if COORDINATES is in WINDOW.\n\
COORDINATES is a cons of the form (X . Y), X and Y being distances\n\
...
If they are on the border between WINDOW and its right sibling,\n\
`vertical-line' is returned.")
(coordinates, window)
register Lisp_Object coordinates, window;
{
int x, y;
CHECK_LIVE_WINDOW (window, 0);
CHECK_CONS (coordinates, 1);
x = XINT (Fcar (coordinates));
y = XINT (Fcdr (coordinates));
switch (coordinates_in_window (XWINDOW (window), &x, &y))
{
case 0: /* NOT in window at all. */
return Qnil;
case 1: /* In text part of window. */
return Fcons (make_number (x), make_number (y));
case 2: /* In mode line of window. */
return Qmode_line;
case 3: /* On right border of window. */
return Qvertical_line;
default:
abort ();
}
}
Note that C code cannot call functions by name unless they are
defined in C. The way to call a function written in Lisp is to use
Ffuncall, which embodies the Lisp function
funcall. Since the Lisp function funcall
accepts an unlimited number of arguments, in C it takes two: the
number of Lisp-level arguments, and a one-dimensional array
containing their values. The first Lisp-level argument is the Lisp
function to call, and the rest are the arguments to pass to it.
Since Ffuncall can call the evaluator, you must
protect pointers from garbage collection around the call to
Ffuncall.
The C functions call0, call1,
call2, and so on, provide handy ways to call a Lisp
function conveniently with a fixed number of arguments. They work
by calling Ffuncall.
`eval.c' is a very good file to look through for examples; `lisp.h' contains the definitions for some important macros and functions.
GNU Emacs Lisp manipulates many different types of data. The actual data are stored in a heap and the only access that programs have to it is through pointers. Pointers are thirty-two bits wide in most implementations. Depending on the operating system and type of machine for which you compile Emacs, twenty-eight bits are used to address the object, and the remaining four bits are used for a GC mark bit and the tag that identifies the object's type.
Because Lisp objects are represented as tagged pointers, it is
always possible to determine the Lisp data type of any object. The
C data type Lisp_Object can hold any Lisp object of
any data type. Ordinary variables have type
Lisp_Object, which means they can hold any type of
Lisp value; you can determine the actual data type only at run
time. The same is true for function arguments; if you want a
function to accept only a certain type of argument, you must check
the type explicitly using a suitable predicate (see section Type Predicates).
Buffers contain fields not directly accessible by the Lisp programmer. We describe them here, naming them by the names used in the C code. Many are accessible indirectly in Lisp programs via Lisp primitives.
namesave_modifiedmodtimeauto_save_modifiedlast_window_startwindow-start position in
the buffer as of the last time the buffer was displayed in a
window.
undo_listsyntax_table_vdowncase_tableupcase_tablecase_canon_tablecase_eqv_tabledisplay_tablenil if it doesn't have one. See section Display Tables.
markersbacked_upmarkmarkers.
See section The Mark.
mark_activenil if the buffer's mark is
active.
local_var_alistbase_buffernil.
keymapoverlay_centeroverlays_beforeoverlays_afterenable_multibyte_charactersenable-multibyte-characters---either t or
nil.
Windows have the following accessible fields:
framemini_pnil if this window is a minibuffer
window.
bufferdedicatednil if this window is dedicated to its
buffer.
pointmstartforce_startnil, it says that the window
has been scrolled explicitly by the Lisp program. This affects what
the next redisplay does if point is off the screen: instead of
scrolling the window to show the text around point, it moves point
to a location that is on the screen.
last_modifiedmodified field of the window's buffer, as of
the last time a redisplay completed in this window.
last_pointlefttopheightwidthnextnil in a window that is the rightmost or
bottommost of a group of siblings.
prevnil in a window that is the leftmost
or topmost of a group of siblings.
parenthscrolluse_timeget-lru-window uses this field.
display_tablenil if none is
specified for it.
update_mode_linenil means this window's mode line needs to be
updated.
base_line_numbernil. This is used for displaying the line number of
point in the mode line.
base_line_posnil meaning none is known.
region_showingnil.
The fields of a process are:
namecommandfilternil.
sentinelnil.
bufferpidchildpnil if this is really a child process.
It is nil for a network connection.
markkill_without_querynil, killing Emacs while this
process is still running does not ask for confirmation about
killing the process.
raw_status_lowraw_status_highwait system call.
statusprocess-status should
return it.
tickupdate_tickpty_flagnil if communication with the subprocess uses
a PTY; nil if it uses a pipe.
infdoutfdsubttynil.)
tty_namenil if it is using pipes.