CMSC 421 Operating Systems Lecture Notes
(c) 1993 Howard E. Motteler


Distributed Systems (Chapter 9)
================================

Hardware (9.2)
---------------

There are a number of ways of classifying systems with more than
one processor; for example

  - shared vs distrib memory
  - tightly vs loosely coupled
  - SIMD vs MIMD


Shared Memory 
--------------

Features of a shared memory system:

   - single common address space
     (all processors share the same memory)

   - usually bus based and synchronous 
     (i.e., has a global clock)

   - communication is through the memory

   - tightly coupled

   - semaphores, TSL, etc., used for sync

Examples: multi-headed Cray & SGI machines

Shared memory advantages

   - familiar programming paradigm (sem's, etc.)

   - easy to place several processors on a bus

Shared memory disadvantages

   - does not scale well, as the bus bandwidth becomes a bottleneck

   - shared data can also be a `logical bottleneck', as a single 
     memory address can be accessed by only one processor at a time

Shared memory systems normally use a memory cache to improve
performance 

This speeds access time and allows a single address to be read
by several processors at the same time	

Problem: cache coherence

   - processors P1 and P2 both have variable V in their cache

   - P1 modifies V

   - P2 won't see the updated value until P1's cache is
     written back to main memory

Solution:

   - use write-through cache 
     (writes are copied back to main memory)

   - snoop: when a cache sees a write to a main memory address 
     it has, it either updates the value or just removes it from 
     the cache


Distributed Memory 
-------------------

In a distributed memory system

   - each processor has its own memory

   - communication can either be bus-based or
     through some kind of message-passing network  
     (text calls this a `switched' system)

Examples: MasPar, Transputers, Intel Paragon, Connection Machines

Some distributed memory machines (e.g., the KSR-1, Cray T3D)
*simulate* shared memory with distributed memory

Message passing distributed memory systems can be further
classified by their geometry (connection scheme)

   - bus-based   (workstations on ethernet)
   - grids       (MasPar X-grid)
   - hypercubes  (CM-2, N-cube)
   - butterfly   (MasPar omega-net, BBN `butterfly')
   - hybrid      (e.g., MasPar uses X-grid and `omega-net')

Important issues for the geometry of a system are

   - valence (number of connections to each processor)
     (grid has fixed valence, hypercube does not)

   - longest path (= longest message time)

   - virtual geometries 
     (e.g., use small grid to simulate a larger one)

   - fixed or varying geometry via a programmable switch


`Loose' and `Tight' Coupling
-----------------------------

Very generally, 

   - tightly coupled = high bandwidth between processors
   - loosely coupled  = low bandwidth between processors

(bandwidth = bytes/second transfer rate)

All the examples discussed so far, except workstations on
ethernet, are tightly coupled

Tightly coupled systems can be used for a broad range of
single applications that need to be run fast

Loosely coupled systems have more limited performance but are
fine for many applications, e.g.,

   - workstations in a lab (LAN)
   - Network File System (NFS)
   - the internet (WAN)


Classification by control stream
---------------------------------

SIMD  ``single instruction multiple data''

   -  (also called `data parallel' machines)

   - One control stream applies to all processors

   - Local data can vary

   - Instructions can specify communication with specific 
     neighbors, e.g., `send x to north neighbor' for a grid

   - (Different geometries have different neighbor sets)

   - canonical programming construct: ``plural''

Examples: MasPar, CM2, various array processors

[ do MPL programming example ]


MIMD  ``multiple instruction multiple data''

   - Each processor has its own control stream

   - More general, possibly harder to program than SIMD

   - Canonical programming construct: PAR, or fork, or some 
     way to start a set of communicating processes

   - CSP and Dataflow are MIMD parallel programming paradigms

   - Dataflow machines are a specific subclass of MIMD machines

Examples: Intel Paragon, Transputers, SGI Challenge

[ do threads or dataflow programming example ]


Software (9.3)
===============

Operating Systems
------------------

Distributed operating systems can be roughly classified as

   - loosely coupled 
     (e.g., Unix with NFS)

   - tightly coupled
     (e.g., SGI Challenge, MasPar, Cray T3D


 [ NFS Details in next note set ]