On this page:
2.1 Using #lang marketplace and friends
2.2 Using Marketplace as a library
ground-vm
ground-vm:
2.3 Constructing transitions
transition
transition:
transition/  no-state
transition
Transition
Action  Tree
Constreeof
sequence-actions
2.4 Creating endpoints
publisher
publisher:
subscriber
subscriber:
observe-subscribers
observe-subscribers:
observe-publishers
observe-publishers:
observe-subscribers/  everything
observe-subscribers/  everything:
observe-publishers/  everything
observe-publishers/  everything:
build-endpoint
build-endpoint:
2.4.1 Receiving messages
on-message
2.4.2 Action-only vs. State updates
match-state
2.4.3 Handling presence and absence events
on-presence
on-absence
2.4.4 Exit reasons
match-reason
2.4.5 Updating endpoints
2.4.6 Who am I talking to?
match-orientation
match-conversation
match-interest-type
2.4.7 Participating in a conversation vs. observing conversations
2.4.8 Naming endpoints
name-endpoint
let-fresh
2.5 Deleting endpoints
delete-endpoint
2.6 Sending messages and feedback
send-message
send-feedback
2.7 Creating processes
spawn
spawn/  continue
spawn:
spawn/  continue:
name-process
2.8 Exiting and killing processes
quit
2.9 Cooperative scheduling
yield
yield:
2.10 Creating nested VMs
spawn-vm
spawn-vm:
2.11 Relaying across layers
at-meta-level:
at-meta-level
6.0.0.1

2 High-level interface

This high-level interface between a VM and a process is analogous to the C library interface of a Unix-like operating system. The Low-level interface corresponds to the system call interface of a Unix-like operating system.

2.1 Using #lang marketplace and friends

 #lang marketplace package: marketplace

Programs written for Marketplace differ from normal Racket modules only in their selection of language. A Racket module written with #lang marketplace, such as the echo server in TCP echo server, specifies a sequence of definitions and startup actions for an application. Typically, initial actions spawn application processes and nested VMs, which in turn subscribe to sources of events from the outside world.

At present, there’s just #lang marketplace. In future, there will be a variation for Typed Racket, and languages providing greater support for flow control, responsibility transfer, and other networking concepts. For now, Typed Racket programs must be written as #lang typed/racket programs using (require marketplace) and ground-vm: explicitly.

2.2 Using Marketplace as a library

 (require marketplace/sugar-untyped)
 (require marketplace/sugar-typed)
  package: marketplace

Instead of using Racket’s #lang feature, ordinary Racket programs can use Marketplace features by requiring Marketplace modules directly.

Such programs need to use ground-vm/ground-vm: to start the ground-level VM explicitly. They also need to explicitly start any drivers they need; for example, the file "examples/echo-plain.rkt" uses ground-vm along with tcp and an initial endpoint action:

(ground-vm tcp
           (subscriber (tcp-channel ? (tcp-listener 5999) ?)
             (match-conversation (tcp-channel from to _)
               (on-presence (spawn (echoer from to))))))

syntax

(ground-vm maybe-boot-pid-binding maybe-initial-state initial-action ...)

syntax

(ground-vm: maybe-boot-pid-binding maybe-typed-initial-state initial-action ...)

 
maybe-boot-pid-binding = 
  | #:boot-pid id
     
maybe-initial-state = 
  | #:initial-state expr
     
maybe-typed-initial-state = 
  | #:initial-state expr : type
     
initial-action = expr
Starts the ground VM, in untyped and typed programs, respectively. If #:boot-pid is specified, the given identifier is bound within the form to the PID of the primordial process that performs the initial actions. If #:initial-state is specified (with a type, for ground-vm:), it is used as the initial state for the primordial process; if it is not supplied, the primordial process is given (void) as its initial state (and Void as its state type).

2.3 Constructing transitions

syntax

(transition new-state action-tree ...)

syntax

(transition: new-state : State action-tree ...)

syntax

(transition/no-state action-tree ...)

Each of these forms produces a Transition structure. The first is for untyped code, the second for typed code (where the mandatory State is the type of the transitioning process’s private state), and the third for either.

Each action-tree must be an (ActionTree State).

It’s fine to include no action-trees, in which case the transition merely updates the state of the process without taking any actions.

In the case of transition/no-state, the type Void and value (void) is used for the process state. transition/no-state is useful for processes that are stateless other than the implicit state of their endpoints.

struct

(struct transition (state actions)
  #:transparent)
  state : State
  actions : (ActionTree State)

type

Transition : (All (State) (transition State))

A transition structure. The transition-state field is the new private state the process will have after the transition is applied, and the transition-actions are the actions that will be performed by the VM in order to apply the transition.

type

ActionTree : (All (State) (Constreeof (Action State)))

type

Constreeof : (All (X) (Rec CT (U X (Pairof CT CT) False Void Null)))

An action-tree is a cons-tree of Actions. When performing actions, a VM will traverse an action-tree in left-to-right order.

'(), (void), and #f may also be present in action-trees: when the VM reaches such a value, it ignores it and continues with the next leaf in the tree.

For example, all of the following are valid action trees which will send messages 1, 2 and 3 in that order:

(list (send-message 1)
      (send-message 2)
      (send-message 3))
(list (list (send-message 1))
      (cons (send-message 2) (cons '() (send-message 3))))
(cons (cons (send-message 1)
            (send-message 2))
      (list #f #f (send-message 3)))

Because #f and (void) are valid, ignored, members of an action-tree, and and when can be used to selectively include actions in an action-tree:

(list (first-action)
      (when (condition?)
        (optional-action))
      (final-action))
(list (first-action)
      (and (condition?)
           (optional-action))
      (final-action))

Finally, these inert placeholders can be used to represent "no action at all" in a transition:

(transition new-state) ; No action-trees at all
(transition new-state '())
(transition new-state (void))
(transition new-state #f)

procedure

(sequence-actions initial-transition    
  item ...)  (Transition State)
  initial-transition : (Transition State)
  item : 
(U (ActionTree State)
   (State -> (Transition State)))
Returns a transition formed from the initial-transition extended with new actions, possibly updating its carried state. Each of the supplied items is examined: if it is an ActionTree, it is appended to the pending transition’s actions; if it is a procedure, it is called with the state of the pending transition, and is expected to return an updated transition.

For example,

(sequence-actions (transition 'x
                    (send-message (list 'message 0)))
                  (send-message (list 'message 1))
                  (send-message (list 'message 2))
                  (lambda (old-state)
                    (transition (cons 'y old-state)
                      (send-message (list 'message 3))))
                  (send-message (list 'message 4)))

produces the equivalent of

(transition (cons 'y 'x)
  (send-message (list 'message 0))
  (send-message (list 'message 1))
  (send-message (list 'message 2))
  (send-message (list 'message 3))
  (send-message (list 'message 4)))

2.4 Creating endpoints

The primitive action that creates new endpoints is add-endpoint, but because endpoints are the most flexible and complex point of interaction between a process and its VM, a collection of macros helps streamline endpoint setup.

syntax

(publisher topic handler ...)

syntax

(publisher: State topic handler ...)

syntax

(subscriber topic handler ...)

syntax

(subscriber: State topic handler ...)

syntax

(observe-subscribers topic handler ...)

syntax

(observe-subscribers: State topic handler ...)

syntax

(observe-publishers topic handler ...)

syntax

(observe-publishers: State topic handler ...)

syntax

(observe-subscribers/everything topic handler ...)

syntax

(observe-subscribers/everything: State topic handler ...)

syntax

(observe-publishers/everything topic handler ...)

syntax

(observe-publishers/everything: State topic handler ...)

syntax

(build-endpoint pre-eid role handler ...)

syntax

(build-endpoint: State pre-eid role handler ...)

The many variations on the core build-endpoint/build-endpoint: form exist to give good control over InterestType in the endpoint under construction; see Participating in a conversation vs. observing conversations.

Almost everything is optional in an endpoint definition. The only mandatory part is the topic, unless you’re using Typed Racket, in which case the process state type must also be specified.

For example, a minimal endpoint subscribing to all messages would be:

(subscriber ?)

or in Typed Racket, for a process with Integer as its process state type,

(subscriber: Integer ?)

A minimal publishing endpoint would be:

(publisher ?)
(publisher: Integer ?)

While topic patterns are ordinary Racket data with embedded ? wildcards (see Messages and Topics), all the other patterns in an endpoint definition are match-patterns. In particular note that ? is a wildcard in a topic pattern, while _ is a wildcard in a match-pattern.

2.4.1 Receiving messages

syntax

(on-message [pattern expr ...] ...)

Supply an on-message handler clause to an endpoint definition to handle incoming message events (as distinct from presence- or absence-events).

The following endpoint subscribes to all messages, but only handles some of them:

(subscriber ?
  (on-message
   ['ping (send-message 'pong)]
   ['hello (list (send-message 'goodbye)
                 (quit))]))
2.4.2 Action-only vs. State updates

syntax

(match-state pattern handler ...)

If a group of handlers is wrapped in match-state, then all the wrapped handlers are expected to return transition structures.

If not, however, the handler expressions are expected to return plain ActionTrees.

This way, simple handlers that do not need to examine the process state, and simply act in response to whichever event triggered them, can be written without the clutter of threading the process state value through the code.

For example, a simple endpoint could be written either as

(subscriber 'ping
  (on-message ['ping (send-message 'pong)]))

or, explicitly accessing the endpoint’s process’s state,

(subscriber 'ping
  (match-state old-state
    (on-message ['ping (transition old-state
                         (send-message 'pong))])))
2.4.3 Handling presence and absence events

syntax

(on-presence expr ...)

syntax

(on-absence expr ...)

Other endpoints (in this or other processes) may have matching topics and complementary orientations to the current endpoint. When such endpoints come and go, presence and absence events are generated in the current endpoint.

By default, no actions are taken on such events, but on-presence and on-absence handlers override this behaviour.

For example, say process A establishes the following endpoint:

(subscriber 'ping
  (on-presence (send-message 'pinger-arrived))
  (on-absence  (send-message 'pinger-departed))
  (on-message ['ping (send-message 'pong)]))

Some time later, process B takes the following endpoint-establishing action:

(let-fresh (ping-endpoint-name pong-waiter-name)
  (name-endpoint ping-endpoint-name
    (publisher 'ping
      (on-presence
       (list (name-endpoint pong-waiter-name
               (subscriber 'pong
                 (on-message
                  ['pong (list (delete-endpoint ping-endpoint-name)
                               (delete-endpoint pong-waiter-name))])))
             (send-message 'ping))))))

The sequence of events will be:

  1. Process A’s on-presence handler will run, and the 'pinger-arrived message will be sent. At the same time,In the current implementation, one happens before the other, but it is nondeterministic which is run first. process B’s on-presence handler runs, installing a second endpoint and sending the 'ping message.

  2. Process A’s endpoint receives the 'ping message, and sends the 'pong message.

  3. Process B’s second endpoint receives the 'pong message, and deletes both of process B’s endpoints.

  4. The on-absence handler in process A runs, sending the 'pinger-departed message.

One possible trace of messages in the VM containing processes A and B is

'pinger-arrived
'ping
'pong
'pinger-departed

By sending the 'ping message only once the on-presence handler has fired, process B ensures that someone is listening for pings.

This way, if process B starts before process A, then B will automatically wait until A is ready to receive ping requests before issuing any.

2.4.4 Exit reasons

syntax

(match-reason pattern handler ...)

If a handler is wrapped in a match-reason form, then the exit reason supplied to the delete-endpoint or quit action that led to the absence-event is available to the endpoint’s on-absence handler expression.

2.4.5 Updating endpoints

If, when an endpoint is created, an existing endpoint with an equal? name is already present, then if the existing and to-be-added endpoints have exactly equal roles (meaning equal orientations, interest-types, and topic patterns), the handlers for the endpoint are updated without emitting presence or absence notifications.

This dubious feature can be used to avoid "glitching" of presence signals. A future release of this library will include better automatic support for avoiding such transients.

2.4.6 Who am I talking to?

syntax

(match-orientation pattern handler ...)

syntax

(match-conversation pattern handler ...)

syntax

(match-interest-type pattern handler ...)

Wrapping a handler in match-orientation, match-conversation, and/or match-interest-type gives a handler access to the contents of the role structure carried in the triggering EndpointEvent.

The carried role describes the intersection of interests between the current endpoint and the peer endpoint, and so can proxy for the identity of the other party. It is in a sense a description of the scope of the current conversation.

It is most common to simply use match-conversation to extract the role-topic alone, since it is seldom necessary to examine role-orientation (since it’s guaranteed to be complementary to the orientation of the current endpoint) or role-interest-type.

See Examples for examples of the use of match-conversation and friends.

2.4.7 Participating in a conversation vs. observing conversations

The core build-endpoint form takes an expression evaluating to a role, rather than a simple topic. This gives full control over the new endpoint’s Orientation and InterestType.

The other forms exist for convenience, since usually the orientation and interest-type is known statically, and only the topic varies dynamically:

The publisher, observe-subscribers and observe-subscribers/everything forms create publisher-oriented endpoints, and subscriber, observe-publishers and observe-publishers/everything create subscriber-oriented endpoints. The rationale for this is that as a participant, the code should declare the role being played; but as an observer, the code should declare the roles being observed.

2.4.8 Naming endpoints

Endpoint names can be used to update or delete endpoints.

procedure

(name-endpoint id add-endpoint-action)  AddEndpoint

  id : Any
  add-endpoint-action : AddEndpoint
Returns a copy of the passed-in add-endpoint action structure, with the id field set to the passed-in identifying value.

syntax

(let-fresh (identifier ...) expr ...)

Binds the identifiers to freshly-gensymmed symbols so that they are available to the exprs. let-fresh is useful for inventing a guaranteed-unused name for a temporary endpoint:

(let-fresh (my-name)
  (name-endpoint my-name
    (subscriber ?
      (on-message [_ (list (delete-endpoint my-name)
                           ...)]))))

2.5 Deleting endpoints

procedure

(delete-endpoint id [reason])  Action

  id : Any
  reason : Any = #f
Use this action to delete a previously-added endpoint by name. The id given must be equal? to the corresponding add-endpoint-pre-eid; when endpoint was used to construct the endpoint to be deleted, the relevant name is that bound by #:let-name or supplied to #:name. See Naming endpoints.

If reason is supplied, it is included in the corresponding action, and made available in any resulting absence-events.

2.6 Sending messages and feedback

procedure

(send-message body [orientation])  Action

  body : Any
  orientation : Orientation = 'publisher
Constructs a message-sending action with the given orientation. Usually the correct orientation to use is 'publisher; it means that the sender of the message is acting in the "publisher" role. Use 'subscriber instead when acting in the "subscriber" role, i.e. sending feedback.

procedure

(send-feedback body)  Action

  body : Any
Equivalent to (send-message body 'subscriber).

2.7 Creating processes

syntax

(spawn maybe-pid-binding boot-expr)

syntax

(spawn/continue maybe-pid-binding
                #:parent parent-state-pattern k-expr
                #:child boot-expr)

syntax

(spawn: maybe-pid-binding
        #:parent : ParentStateType
        #:child : ChildStateType boot-expr)

syntax

(spawn/continue: maybe-pid-binding
                 #:parent parent-state-pattern : ParentStateType k-expr
                 #:child : ChildStateType boot-expr)
 
maybe-pid-binding = 
  | #:pid identifier
     
k-expr = expr
     
boot-expr = expr
Action describing a new process to create. The boot-expr should be an expression yielding a transition that contains the child process’s initial state and initial actions.

If #:pid is supplied, the associated identifier is bound to the child process’s PID in both boot-expr and the parent’s k-expr.

The spawn/continue and spawn/continue: variations include a k-expr, which will run in the parent process after the child process has been created. Note that k-expr must return a Transition, since parent-state-pattern is always supplied for these variations.

In Typed Racket, for type system reasons, spawn: and spawn/continue: require ParentStateType to be supplied as well as ChildStateType.

procedure

(name-process id spawn-action)  Spawn

  id : Any
  spawn-action : Spawn
Returns a copy of the passed-in spawn action structure, with the debug-name field set to the passed-in identifying value. The debug name of a process is used in VM debug output. See also logging (MARKETPLACE_LOG).

2.8 Exiting and killing processes

procedure

(quit [who reason])  Action

  who : (Option PID) = #f
  reason : Any = #f
Action causing the termination of a process. If who is omitted or #f, terminates the acting process; otherwise, terminates the peer process having who as its PID.

If reason is supplied, it is included in the corresponding action, and made available in any resulting absence-events.

Terminating the current process is as simple as:

(quit)

When a process raises an exception that it does not catch, its containing VM catches the exception and turns it into an implicit quit action. In that case, the reason will be the raised exception itself.

2.9 Cooperative scheduling

syntax

(yield state-pattern k-expr)

syntax

(yield: state-pattern : State k-expr)

Lets other processes in the system run for a step, returning to evaluate k-expr only after doing a complete round of the scheduler.

The state of the yielding process will be matched against state-pattern when the process is resumed, and k-expr must evaluate to a Transition.

2.10 Creating nested VMs

syntax

(spawn-vm maybe-vm-pid-binding maybe-boot-pid-binding
          maybe-initial-state
          maybe-debug-name
          boot-action-expr ...)

syntax

(spawn-vm: : ParentStateType
           maybe-vm-pid-binding maybe-boot-pid-binding
           maybe-typed-initial-state
           maybe-debug-name
           boot-action-expr ...)
 
maybe-vm-pid-binding = 
  | #:vm-pid identifier
     
maybe-boot-pid-binding = 
  | #:boot-pid identifier
     
maybe-initial-state = 
  | #:initial-state expr
     
maybe-typed-initial-state = 
  | #:initial-state expr : StateType
     
maybe-debug-name = 
  | #:debug-name expr
     
boot-action-expr = expr
Results in a spawn action that starts a nested VM. The primordial process in the new VM executes the boot-actions with the given initial state. (If no initial state is supplied, (void) is used.)

If #:vm-pid is present, the corresponding identifier is bound in the boot-action expressions to the container-relative PID of the new VM itself. If #:boot-pid is present, however, the corresponding identifier is bound to the new-VM-relative PID of the primordial process in the new VM.

2.11 Relaying across layers

syntax

(at-meta-level: StateType preaction ...)

procedure

(at-meta-level preaction ...)  (Action StateType)

  preaction : (PreAction State)
Each VM gives its processes access to two distinct IPC facilities: the internal one, provided for the VM’s processes to talk amongst themselves, and the external one, the network that the VM itself is a process within.

Marketplace’s actions can apply to either of those two networks. By default, actions apply to the VM of the acting process directly, but using at-meta-level (or at-meta-level: in typed code) to wrap an action level-shifts the action to make it apply at the level of the acting process’s VM’s container instead.

For example, wrapping an endpoint in at-meta-level adds a subscription to the VM’s container’s network. Instead of listening to sibling processes of the acting process, the new endpoint will listen to sibling processes of the acting process’s VM. In this example, the primordial process in the nested VM creates an endpoint in the VM’s own network, the ground VM:

(spawn-vm
 (at-meta-level
  (subscriber (tcp-channel ? (tcp-listener 5999) ?) ...)))

In this example, a new process is spawned as a sibling of the nested VM rather than as a sibling of its primordial process:

(spawn-vm
 (at-meta-level
  (spawn (transition/no-state (send-message 'hello-world)))))

Compare to this example, which spawns a sibling of the nested VM’s primordial process:

(spawn-vm
 (spawn (transition/no-state (send-message 'hello-world))))