Structure of emacs text

The effective structure of emacs text

Emacs wants to structure text

Emacs is not a structural editor, but it structures text a lot. Right now I'm writing in org-mode, and in this one buffer I'm looking at maybe 3 dozen distinct meaningful regions displaying in 8 different faces. And that's just what I've written so far, here at the third paragraph.

Furthermore, there are outlines, ewocs, widgets, overlays, and stretches of text that have their own keymaps, or faces, or actions before and after insertion, etc. On the emacs-devel list, they were recently proposing "islands" of text that would nearly have their own modes.

But the structure is a trick

Yet except for overlays, they are all tricks. They are faked by careful control over emacs' interaction with the text "inside" the fake objects. This can involve controlling point motion, regions, insertion, marks, display, sticky properties, etc. As an elisp programmer, IMO this need to fake structure contributes more complexity to elisp code than any other single factor does.

It also makes for fragile interactions. A user can accidentally transgress the structural assumptions, for instance by splitting a headline in an outline. Sometimes I do. It's hard to prevent.

Flexibility is important too

On the other hand, there's something to be said for that situation. It's flexible. It doesn't limit you to a predefined set of types and operations on them. It doesn't need to predict what sort of structure will be useful, because it doesn't provide any. It just hands the Elisp programmer a set of basic tools with which to build structure.

The dimensions of useful text structure

Boundaries

Solid objects

On the one hand, there are structures that want to be real objects, with an inside and an outside. For example, an ewoc or a widget.

There's no such thing as half of one of these. Half an ewoc makes no sense. If you kill half the text inside a widget, you're left with a complete widget that has less text.

We don't want to cross scope on these objects at all. A region with (say) point inside a widget and mark outside is not generally useful.

Not solid: Mere stretches of text

On the other hand, there are structures that want to be just stretches of text. For example, ordinary text even if you have marked a region in it, or given some part of it a special face or property.

These have no intrinsic inside and outside, nor any strong intrinsic structure. In fact, these things are scarcely "things". We're just pointing at some length of text, asking whether it's an object, and getting "no".

Semi-solid objects

And there are intermediate cases. For example, an item in an outline. For a less solid example, a word or paragraph in a text mode.

These still want to have an inside and an outside, but it's editable. If you delete the whitespace between two words, you've got just one word.

Holds data?

If there's a text-containing object, can it hold data in addition to the text itself? In emacs, the answer is always "yes". Text can have properties, which can be of any type.

So the question is not "does it hold data?", but "does it hold data in a way that covaries with the (pseudo)type of object it is?"

For ewocs and widgets, the answer is clearly yes. Even for outline items, the answer wants to be yes. For instance, org items can have properties, and all outline items have an implied "depth" property.

Has behavior?

Again, in emacs the answer is always "yes", and of course there's the caveat that behavior is really due to commands and not objects and we need to rule out buggy commands that don't understand the given object.

So again we need to refine the question. "Does it, in conjunction with the set of commands appropriate to it, behave in a way that covaries with the (pseudo)type of object it is?"

Again, for the solid objects the answer is clearly yes, and "yes" also seems correct for the semi-solid objects.

Weight

Lightweight

For instance, words and paragraphs. Emacs deals specially with those in various ways. But it often does so without remembering the lightweight object as an entity. For instance, word constituents are defined the syntax table.

Heavyweight

For instance, widgets. Also ewocs.

In between

Of course there's a whole spectrum of weight in between widgets and words.

Impact on buffer structure

Observation

(This is not a category). The other dimensions were keyed by object type, but for this dimension ISTM it makes more sense to key by mode type.

Total

In some modes, the buffer wants to be structured from beginning to end. For instance, dired or gnus.

Yanking in unstructured text would just confuse the mode. Often the buffer is read-only, even if user operations modify it, so that one can't mess up the structured text at all.

Minimal

Some modes expect essentially no structure. For instance, fundamental mode. One can yank in arbitrary text; it's nothing special. Now the buffer has more text in it.

Text-mode is another example, now a little more structured. It can be viewed as structured into paragraphs, but any text whatsoever qualifies as zero or more paragraphs.

Partly structured

Other modes are intermediate. For instance, outline mode, org mode, or almost any source code mode.

These modes generally try to cope with any arbitrary text in it. They don't try to prevent killing or yanking. But they also treat some text specially or give it extra meaning - for outline, it's headlines and stars. For source code, it generally includes comments, code, strings.

So the user is free to kill or yank, but needs to be somewhat careful and needs to understand the meaning of various types of text.

The picture

So as I see it, the picture is one of wanna-be objects of varying sizes floating around in emacs buffers. The heavier ones are trying to be real objects, the lighter ones aren't (much). Some modes want to be made up of text, others really want to be a list or tree of objects.

In the next post, I plan to build on that.