---
author: ''
category: ''
date: 2007/06/20 13:23
description: ''
link: ''
priority: ''
slug: BB597
tags: programming
title: 'Old READMEs: Atlast... make everything programmable!'
type: text
updated: 2007/06/20 13:23
url_type: ''
---
I have been exploring embeddable languages for the last month or so. I have learned forth and some of its many *many* **many** variants [1]_ and while exploring `one of the most obscure ones`_ called Atlast [2]_, I found a very interesting README_ which I will quote liberally in this post.

    Virtually  every  industry  analyst  agrees  that open architecture is
    essential to the success of applications.  And  yet,  even  today,  we
    write  program  after  program  that  is closed--that its users cannot
    program--that admits of no extensions without our adding to its source
    code.  If we believe intellectually, from a sound understanding of the
    economic incentives in the marketplace, that open systems are  better,
    and  have  seen  this  belief confirmed repeatedly in the marketplace,
    then the only question that  remains  is  why?   Why  not  make  every
    program an open program?

And this is indeed very important. Any app worth developing is probably worth 
developing in an extensible manner. After all, you are probably never going to 
figure out what the user really wants to do.

But why is not every program "open" in this manner?

    Well,  because  it's HARD!  Writing a closed program has traditionally
    been much less work at every stage of the development cycle: easier to
    design,  less  code  to  write,  simpler  documentation, and far fewer
    considerations in the test phase.  In addition,  closed  products  are
    believed  to  be  less demanding of support, although I'll argue later
    that this assumption may be incorrect.

This is true, although it was much more true in 1991 when Atlast was released to 
the public domain. Nowadays the generalized adoption of more dynamic languages 
has made this much easier. After all, you can write a plugin system for your python app
in perhaps ten lines of code.

However, there is a strong backlash from the pains of MSOffice scripting, which involved 
loveliness like function names that changed according to your locale, and basic as
the language of choice.

On static languages this is harder, but there are technological solutions. For example, in
KDE there is already a limited scriptability culture based on how easy it is to provide 
external DCOP interfaces (I suppose I should say DBUS nowadays). Also for KDE4 the Kross
scripting framework promises language-neutral scripting.

    Most programs start out as nonprogrammable, closed applications,  then
    painfully  claw  their way to programmability through the introduction
    of a limited script or macro facility, succeeded  by  an  increasingly
    comprehensive  interpretive  macro language which grows like topsy and
    without a coherent design as user  demands  upon  it  grow.   Finally,
    perhaps,  the program is outfitted with bindings to existing languages
    such as C.

Noone provides straight custom C bindings for applications now, right? But yes,
I expect scripting is not the first thing that gets implemented.

    An alternative to this is adopting a standard language  as  the  macro
    language  for  a product.  This approach has many attractions.  First,
    choosing a standard language  allows  users  to  avail  themselves  of
    existing  books  and  training  resources  to  learn  its basics.  The
    developer of a dedicated macro language must create all this  material
    from  scratch.   Second,  an interpretive language, where all programs
    are represented in ASCII code, is inherently portable across computers
    and  operating  systems.   Once the interpreter is gotten to work on a
    new system, all the programs it supports are pretty much guaranteed to
    work.   Third,  most existing languages have evolved to the point that
    most of the rough edges have been taken off their  design.   Extending
    an  existing  language  along  the lines laid down by its designers is
    much less likely  to  result  in  an  incomprehensible  disaster  than
    growing an ad-hoc macro language feature by neat-o feature.
    
..

    Unfortunately,   interpreters   are   slow,   slow,  slow.   A  simple
    calculation of the number of instructions of overhead per  instruction
    that  furthers  the execution of the program quickly demonstrates that
    no interpreter is suitable for serious computation.  As  long  as  the
    interpreter  is deployed in the role of a macro language, this may not
    be a substantial consideration.  However, as soon as applications  try
    to  do substantial computation, the overhead of an interpreter becomes
    a crushing burden, verging on intolerable.  The obvious alternative is
    to provide a compiled language.  But that, too, has its problems.

Interesting point here is that again, the speed of interpreters is not so 
big a deal nowadays. After all, LuaJIT_ is pretty fast for most uses.

Using a compiler as an extension language is not something I have run into
but I could do some pretty things using, for example, TCC_, Python and 
python-instant_.

Now we get to ATLAST itself:

    ATLAST   is   a   toolkit   that   makes   applications  programmable.
    Deliberately designed to be  easy  to  integrate  both  into  existing
    programs  and  newly-developed  ones, ATLAST provides any program that
    incorporates it most of the  benefits  of  programmability  with  very
    little explicit effort on the part of the developer.  Indeed, once you
    begin to "think ATLAST" as part of the design cycle,  you'll  probably
    find that the way you design and build programs changes substantially.
    I'm coming to think of ATLAST as the "monster that feeds on programs,"
    because  including  it  in  a  program  tends  to shrink the amount of
    special-purpose code that would otherwise have  to  be  written  while
    resulting in finished applications that are open, extensible, and more
    easily adapted to other  operating  environments  such  as  the  event
    driven paradigm.
  
..
  
    The  idea  of  a  portable  toolkit, integrated into a wide variety of
    products, all of which thereby share  a  common  programming  language
    seems  obvious once you consider its advantages.  It's surprising that
    such packages aren't commonplace in the industry.  In fact,  the  only
    true  antecedent  to  ATLAST I've encountered in my whole twisted path
    through this industry was the universal macro package developed in the
    mid  1970's  by  Kern  Sibbald  and  Ben Cranston at the University of
    Maryland.  That package, implemented on Univac mainframes, provided  a
    common  macro  language  shared  by  a  wide  variety of University of
    Maryland utilities, including a text editor,  debugger,  file  dumper,
    and  typesetting  language.   While  ATLAST  is  entirely different in
    structure and operation  from  the  Maryland  package,  which  was  an
    interpretive  string  language,  the  concept of a cross-product macro
    language and appreciation of the  benefits  to  be  had  from  such  a
    package are directly traceable to those roots.

This concept was later adopted by Lua, Tcl, and you **could** use Python 
in this manner, although usually it's done the other way around. Which means
that there is not so much sharing of extension languages as there could be.

And onto the conclusions:

    Everything  should  be  programmable.  EVERYTHING!  I have come to the
    conclusion that to write almost any program in a closed  manner  is  a
    mistake that invites the expenditure of uncounted hours "enhancing" it
    over its life cycle.  Further tweaks, "features,"  and  "fixes"  often
    result  in  a  product so massive and incomprehensible that it becomes
    unlearnable, unmaintainable, and eventually unusable.

Amen. 

    Far better to invest the effort up front to create a product  flexible
    enough  to be adapted at will, by its users, to their immediate needs.
    If the  product  is  programmable  in  a  portable,  open  form,  user
    extensions  can  be  exchanged,  compared,  reviewed  by  the  product
    developer, and eventually incorporated  into  the  mainstream  of  the
    product.

This prefigures the current FLOSS ecosystem, which is pretty impressive for 
a product that was **mature** in 1991, although of course there was already
a community of EMACS hacking and similar niches.

    It  is  far,  far better to have thousands of creative users expanding
    the scope of one's product in  ways  the  original  developers  didn't
    anticipate--in fact, working for the vendor without pay, than it is to
    have thousands of frustrated users writing up wish list requests  that
    the  vendor  can  comply with only by hiring people and paying them to
    try  to  accommodate  the  perceived  needs  of   the   users.    

.. role:: raw-html(raw)
   :format: html

True if a little too cynical for my current taste ;-) Of course in FLOSS
there is no such conflict between users and developers because the
developers can usually just :raw-html:`<strike>whine</strike>` explain to 
the users the realities of free software development. But hey, a low barrier 
to entry is always a nice thing to have.

    Open
    architecture  and programmability not only benefits the user, not only
    makes a product better in  the  technical  and  marketing  sense,  but
    confers  a  direct  economic  advantage  upon  the  vendor  of  such a
    product--one mirrored in a commensurate disadvantage to the vendor  of
    a closed product.

..

    The   chief  argument  against  programmability  has  been  the  extra
    investment needed to create open products.  ATLAST provides a  way  of
    building  open  products  in  the same, or less, time than it takes to
    construct closed ones.  Just as no C  programmer  in  his  right  mind
    would  sit  down  and  write  his own buffered file I/O package when a
    perfectly fine one was sitting in the library, why re-invent  a  macro
    language  or  other  parameterisation  and  programming  facility when
    there's one just sitting there that's as fast as native C code for all
    but  the  most  absurd misapplications, takes less than 51K with every
    gew-gaw and optional feature at its command enabled all  at  once,  is
    portable to any machine that supports C by simply recompiling a single
    file, and can be integrated into a  typical  application  at  a  basic
    level in less than 15 minutes?

And then proceeds to throw a Forth variant at you ;-) Good concept, perhaps
not the nicest language to use, although the choice is very understandable.

    Am  I  proposing  that every application suddenly look like FORTH?  Of
    course not; no more than output from PostScript  printers  looks  like
    PostScript,  or  applications  that  run  on 80386 processors resemble
    80386 assembly language.  ATLAST is  an  intermediate  language,  seen
    only by those engaged in implementing and extending the product.  Even
    then, ATLAST is a chameleon which, with properly  defined  words,  can
    look like almost anything you like, even at the primitive level of the
    interpreter.

..
    
    Again and again, I have been faced with design situations where I knew
    that  I  really  needed programmability, but didn't have the time, the
    memory, or the fortitude to face the problem squarely and solve it the
    right  way.   Instead,  I ended up creating a kludge that continued to
    burden me through time.  This is just a higher level manifestation  of
    the  nightmares  perpetrated  by  old-time programmers who didn't have
    access to a proper dynamic memory allocator or  linked  list  package.
    Just  because  programmability is the magic smoke of computing doesn't
    mean we should be spooked by the ghost in the machine or  hesitant  to
    confer its power upon our customers.
    
Oh yes. Libraries rule, and when there is no such thing as the library you 
need, it's the worst position to be in.

    Don't  think  of  ATLAST as FORTH.  Don't think of it as a language at
    all.  The best way to think of ATLAST is as  a  library  routine  that
    gives you *programmability*, in the same sense other libraries provide
    file access, window management, or  graphics  facilities.   The  whole
    concept  of  "programmability  in  a can" is odd--it took me two years
    from the time I first thought about it  until  I  really  got  my  end
    effector around it and crushed it into submission.

I am probably going to use ATLAST or something similar in a program, but 
the user will not see a forth-like language at all, but a classical
Excel-like formula language which would get compiled to the forthish
language behind the scenes.

    Open is better.  ATLAST lets you build open programs in less time than
    you used to spend writing closed ones.  Programs  that  inherit  their
    open  architecture  from  ATLAST will share, across the entire product
    line and among all hardware  platforms  that  support  it,  a  common,
    clean,  and  efficient  means  of  user  extensibility.  The potential
    benefits of this are immense.

Indeed. Probably not using ATLAST, but Lua or Python or something else instead.
And still, 16 years after ATLAST was released to the public domain, we are still
walking down this road, and not even close to the goal.

.. _tcc: http://www.tinycc.org

.. _python-instant: http://www.fenics.org/wiki/Instant

.. _luajit: http://www.luajit.org

.. _README: http://www.fourmilab.ch/atlast/README

.. _one of the most obscure ones: http://www.fourmilab.ch/nav/topics/components.html

.. [1] As they say, "Once you see a forth, you have seen a forth"

.. [2] The "Autodesk Threaded Language Application System Toolkit", by John Walker. Version 1.0 was released in August 1995, 1.1 in July 2002... I expect 1.2 around August 2009 :-)