On this page:
Marketplace:   Network-Aware Programming

Marketplace: Network-Aware Programming

Tony Garnock-Jones <tonyg@ccs.neu.edu>

Every program is a network. This is the insight behind the π-calculus. Encoding a program as a π-calculus term shows it as a network of communicating processes. It is also one of the original inspirations for Smalltalk, where every object, every value, was imagined to be a separate computer in a vast network, and where objects communicated by message-passing.

Every program is part of a network. A program that computes a result but cannot communicate it is useless indeed. Every complete program both computes and communicates. Furthermore, it does so with some finite set of resources, which it must carefully manage.

Our programming languages do not recognise that every program is a network. They blur the distinction between stateful and stateless portions of a program, making it difficult for programmers to reason about concurrency, contention, and distribution. They often treat partial failure as an afterthought, despite its importance in reasoning about program behaviour, particularly in connection with the effect of exceptions on stateful programs. They seldom consider issues of trust and security.

Our programming languages do not recognise that every program is part of a network. They treat communication with the outside world in an ad-hoc manner. They frequently treat network communication separately from 1950s-style terminal input and output. They force the programmer to divine failures in other parts of the network by arcane means such as timeouts and examining the entrails of dead communication channels. They offer no support for allocating or releasing local resources in response to changes in other parts of the network. They seldom consider issues of trust and security.

Marketplace is a network-aware programming language. As a corollary, because every program not only computes but also communicates and manages its resources, Marketplace is also a distributed operating system.

By recognising that programs communicate both internally (between subprograms) and externally (between peers), we recognise an inherently recursive layered architecture. We see at every level the same concerns of resource management, location of mutable state, failure detection and recovery, access control, I/O and user interface, debugging and profiling.

Marketplace addresses these concerns with a small set of primitives chosen to make network programming in-the-small as flexible, scalable, manageable and securable as network programming in-the-large—and vice versa.

    1 Concepts

      1.1 What is a process, what are event handlers?

      1.2 What is a VM?

      1.3 Endpoints: Subscription and Advertisement

      1.4 Messages and Topics

      1.5 Presence

      1.6 Nesting, relaying, and levels of discourse

    2 High-level interface

      2.1 Using #lang marketplace and friends

      2.2 Using Marketplace as a library

      2.3 Constructing transitions

      2.4 Creating endpoints

        2.4.1 Receiving messages

        2.4.2 Action-only vs. State updates

        2.4.3 Handling presence and absence events

        2.4.4 Exit reasons

        2.4.5 Updating endpoints

        2.4.6 Who am I talking to?

        2.4.7 Participating in a conversation vs. observing conversations

        2.4.8 Naming endpoints

      2.5 Deleting endpoints

      2.6 Sending messages and feedback

      2.7 Creating processes

      2.8 Exiting and killing processes

      2.9 Cooperative scheduling

      2.10 Creating nested VMs

      2.11 Relaying across layers

    3 Low-level interface

      3.1 Handler Functions

      3.2 Messages, Topics and Roles

      3.3 Endpoint Events

      3.4 Actions

        3.4.1 Endpoints and Messages

        3.4.2 Process Management

    4 Drivers

      4.1 event-relay

      4.2 tcp-bare

        4.2.1 TCP channels

        4.2.2 TCP addresses

        4.2.3 Opening an outbound connection

        4.2.4 Accepting inbound connections

        4.2.5 Receiving data

        4.2.6 Sending data

      4.3 timer (typed and untyped)

      4.4 udp (typed and untyped)

    5 Management and Monitoring

      5.1 generic-spy

      5.2 logging (MARKETPLACE_LOG)

      5.3 debugger (experimental)

    6 Examples

      6.1 TCP echo server

      6.2 TCP chat server

      6.3 TCP chat client