[Beowulf] small-footprint MS WIn "MinWin"

[Robert G. Brown]

On Mon, 22 Oct 2007, Brian D. Ropers-Huilman wrote:

> On 10/22/07, Robert G. Brown <rgb at phy.duke.edu> wrote:
>> This in turn moves their system closer to being a suitable host for VMs,
>> as I think the ideal towards which OS's are moving is the kernel as a VM
>> host, everything else as a VM guest on top of a device-independent
>> layer.  At least I hope we are going there.
>
> Interesting comment for this particular list. While I'm all in favor
> of MS "seeing the light," so to speak, your comment on "... everything
> else as a VM guest on top of a device-independent layer.  At least I
> hope we are going there." With everything so performance oriented in
> our world, I just find this comment surprising.

I don't really see any "intrinsic" conflict, but let's explore this.  If
we really, truly, wanted to run our programs as fast as they possibly
could, we wouldn't really use "a kernel" at all.  We would write
bootloaders that ran our applications, each one custom compiled for a
very specific hardware platform, directly on the hardware.  That way we
wouldn't waste cycles on silly things like incrementing a timer counter,
managing time slices in multitasking contexts, and so on.  Don't NEED
multitasking.  We're only running this one application, period.  In
fact, if we load the bootloader itself onto (say) paper tape and build a
mechanical sprocket system to load IT triggered by certain hand toggled
sense switches and a button, and loaded the program from e.g. tape using
the bootloaded bootloader, we'd have invented early computers, say a PDP
1 (which I used to boot in just this way).

If we wanted to make a small compromise and perhaps manage a very few,
very standard peripheral devices and maybe make it easier to run
different programs, we might right the smallest possible operating
system -- something capable of taking a portable binary that we want to
run and loading it onto the CPU to run, while providing it with some
more or less fixed entry points into code for e.g. reading or writing
disk, maybe a network device (trickier, because of the asynchronous
problems there), and of course managing memory.  Let's name our creation
"DOS" just for grins.

Well, we discover that while DOS is just gangbusters for a single system
running a single task at a time standalone (no network), as soon as we
interconnect it with other systems, or want it to do more than one thing
"at a time" with some fine grained task switching mechanism, or want to
add to the system a wide range of rapidly varying peripheral devices
with their own communications requirements (some of them asynchronous)
it starts to COST us time.  It still runs our application as fast as
anything but direct loading onto a CPU with no devices other than memory
and something one can use to write out the result to (which might be
code to put simple signals on a twisted pair serial line) but we find
ourselves spending all of our time writing custom code snippets,
managing our network of systems, trying to fit in this new device and
then that one, each one a one-off solution.

It also really COMPLICATES writing complex code, especially code that
requires communications between systems.  To solve this problem, we
invent a real multitasking kernel with a scheduler, we invent interrupts
and put peripheral device code into the kernel that hides the details of
peripheral devices behind a "portable" interface.  The result is almost
certainly is a bit slower than pure inline code would be, but it is a
thousand, a million times faster to develop the code itself and once the
interface is done, one doesn't have to recompile all sorts of
applications to use it.  Let's call this system "multics" or "unix" or
whatever you like.

Well, time passes, and the network goes from being a peripheral device
per se to being an integral part of all computers.  Disks are
standardized at the hardware level to where a single program can manage
a wide range of disks without tweaking.  Many NEW devices are created, a
ferment process of new unique and old standard drivers emerges that is
constantly changing.  Computer UIs change, change, change, becoming
graphical and intricate.  Multitasking is necessary to do the simplest
thing on a computer as it is ALWAYS doing several dozen things with at
least one or two timeslices in any given second.  And the processors,
memory and everything gets FASTER and BIGGER at a breakneck pace.  The
work that originally took DOS quite a long time -- reading in a few
hundred kilobytes of data through a 5 MHz bus -- starts to take such a
small amount of time that it is easy to mistake it for "zero" time.

At the same time the management issue and other high level expenses
continue to skyrocket.  Devices are invented but not uniformly supported
by software.  There are many choices for operating system, but only one
of them gets any attention from hardware vendors because it dominates
the market.  It gets to where it is EASY to run a program on a computer
and get 99% of the CPU if you want it, as mundane operation of the GUI
and editors and this and that takes a small and rapidly shrinking
fraction of 1% of the total system's computing capacity!  Suddenly there
is little advantage to making the computer more efficient -- we could
all achieve that easily by just booting the computer single user or even
single process -- but there are huge costs associated with trying to get
this driver to work for that peripheral, dealing with security and user
access, dealing with software portability and availability.  MANAGEMENT
is eating you alive, not delays caused by the fact that you're running a
complex operating system.

In response to this, the manufacturers of the CORE software (not the
peripheral device makers, who profit from the chaos) decide to make it
really easy to create a new split in the function of the kernel.  They
begin by creating multicore CPUs and CPUs with a generalized "virtual"
operating mode that exploits the cheapness of memory.  This goes hand in
glove with the development of operating system kernels that basically
recapitulate and renegotiate the hardware portability interface, the
reason for having a kernel in the first place.  The cost of developing
and managing computer networks and large bodies of UI software goes way
down if one can engineer the bulk of it on top of virtual devices.  Yes,
there is a small cost associated with this.  THERE ALWAYS HAS BEEN
pretty much ever since DOS.  How large the cost is depends on how
complex the interface is, how much detail is being hidden, but it need
not be too great, actually.

When it's done, what might we expect to see?  Well, I'm hoping for a
layer that for most devices manages those devices, virtualized, at a
"cost" of order 1% more than running the same device the way we do now,
which in turn is order 1% slower than running the device directly in raw
code with no standardized layer or interface.  In other words, we spend
1-2% of potential performance for a really, really big boost in OVERALL
productivity, because VM guests are really rather amazing entities --
they go beyond multitasking programs and permit you to multitask
operating systems.

Is it efficient?  Of course not.  Does it matter?  YMMV, but in many
cases it won't.  EVEN IN THOSE CASES WHERE IT DOES it will almost
certainly prove possible to extend a low level "efficient" interface up
through guest layers, possibly at the cost of dedicating the interface
to the VM in question.

Low cost, high benefit.  That's why I'm all for it.

Now, if I had my perfect druthers I'd RATHER have standardization of all
device interfaces and get rid of the need to load distinct drivers at
all.  This too would have negative consequences -- makes it a lot more
difficult to invent new devices, for example -- but boy would it make
code development and systems management easy on what amounts to a
stationary platform with a fixed list of generic peripherals.  Barring
that, VMs still allow the hardware list to be all chaotic and
everything, but do try to provide that nice compatibility layer.

In summary, I think it would be lovely for the same reason I think Linux
on a modern multicore is lovely compared to a bootloaded single task
engine like the PDP 1.  I can work in Lin, Win, and run a couple of
secured environments off on the side all at once on different cores
inside one system.  I can freely and dynamically reallocate resources
among these operating environments.  I can create rigorously
resource-controlled and PORTABLE environments for even cluster tasks
that don't require me to completely reinstall what may be somebody
else's cluster nodes in order to move my tasks there and run them.  I'm
willing to give up a few cycles to gain these benefits, just as I
wouldn't dream of giving up the benefits of at least level 3 operation
on nodes now.

I'm just curious.  Who runs their cluster nodes at level 1 or 2 (plus
networking as needed)?  Anybody?  Show of hands?

    rgb

>
>

-- 
Robert G. Brown
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone(cell): 1-919-280-8443
Web: http://www.phy.duke.edu/~rgb
Lulu Bookstore: http://stores.lulu.com/store.php?fAcctID=877977