Erights - User contributions [en]

Proposed Asynchronous Control Flow Operators

Const — Thu, 29 Nov 2007 18:54:35 GMT

Const:

TODO possibly there should be per-operator pages.

This page describes some not implemented asynchronous control flow operators that could simplify creation of asynchronous application. The intial set of operators is based on experience of implementing the AsyncObjects framework and Sebyla project design.

== Later ==
This is an extremely simple construct. But is useful when it is required to execute code in this or other vat at some later time. In AsyncObjects this constructs is implemented using AsyncAction class, and it was used quite a lot in applications.
<pre>
“later” ( “(” <vatHandle> “)” )? “->” “{“
<code>
“}”
</pre>
When code is executed in other vat, the runtime checks that code closes only over deep frozen or pass by copy objects. If vat is not specified, the code is scheduled to be executed at later turn in this vat. The vatHandle is an expression that evaluates to capability of executing actions on the vat.

==Using==

This is a very simple construct that helps with resource management. The syntax is like the following:
<pre>
“using” “(“ ( (“def” <varName> “:=” )? <eventualOpenExpression> )+ “)” “->” “{“
<statements>
“}”
</pre>
The operator is executed as the following:
#All resources are opened in the specified order. If expression returns promise, the next expression is not executed before first one complete. If <varName> is specified for resource, the resolved resource is assigned to it and it can be used in subsequent expressions and <statements>.
#If opening of some resource fails, the proposed result of operator is that failure.
#Statements are executed, the proposed result of operator is value of last expression. If the result is promise, the operator waits until it is resolved.
#All opened connections are closed, in reverse order. It includes connections that has not been named. If some close operations fail, the most top level one is returned.
Example:
<pre>
using(def r1 := factory1<-openConnection(), def r2 := factory2<-openConnection(), lock<-lockEnter()) -> {
doSomething(r1,r1) # assuming that lock is being held
}
</pre>
The constructs expands to the following:
<pre>
when(def r1 := factory1<-openConnection()) {
when(def r2:= factory2<-openConnection()) {
when( def tmpVar := lock<-lockEnter()) {
doSomething(r1,r1);
} finally {
tmpVar<-close();
}
} finally {
r2<-close();
}
} finally {
r1<-close();
}
</pre>

==Seq==

The are several form of seq operator.
The basis form is the following.
<pre>
“seq” (“def” <varName> “:=”)? “{“
<statements-1>
“}” ( “then” (“def” <varName> “:=”)? “{“
<statements-i>
“}” ) * ( “catch” “(“ <exceptionVar> “)” “{“
<catchStatements>
“}” )? (“finally” “{“
<finallyStatements>
“}”)?
</pre>
The code is executed like the following:
#The code in <statements-i> is executed sequentially. The next code block is executed only if promise returned from previous code block is resolved. If the block yields near value, the next block is executed on the same turn.
#The result of operator is the value of the last block.
#If some of blocks fails, the next blocks are not executed, and control is passed to the catch section and the result of operator will be the result of executing catch statement. If there is no catchSection, the operator fails with expression.
#If there is a finally statement, it is executed after last block or after catch statement. If it fails, the entire seq expression fails with the same exception.
Example:
<pre>
seq def a := {
doA()
} then {
doSomething(a)
} catch(e) {
print(e)
throw e
} finally {
cleanup()
}
</pre>
Example is expanded as the following:
<pre>
when(someTempResolvedPromise) {
def aVow := {
doA()
}
when (def a := aVow) {
doSomething(a)
}
} catch(e) {
print(e)
throw e
} finally {
cleanup()
}
</pre>
There is also a short form:
<pre>
“seq” “(“ (“def” <varName> “:=” )? <eventualExpression> ( “,” (“def” <varName> “:=” )? <eventualExpression> )* “)”
</pre>
The comma separated expressions are executed in sequence. The named values are available at later expressions.
The are three loops forms:
<pre>
“seq” “do” “{“
<statements>
“}” “while” “(“ <eventualExpressions> “)”

“seq” “while” “(“ <eventualExpressions> “)” “{“
<statements>
“}”

“seq” “for” <pattern> “in” <collection> “{“
<statements>
“}”
</pre>
The semantics is the same as for normal loops. The next iteration starts only after previous completed.
TODO it might be useful to keep intermediate results of previous iteration and to make them available in the body of the loop and conditions. On other hand, it is quite easy to use basic seq operator to achieve the same result.
TODO “seq for” needs support for eventual iterators.

==All==
This operator allows execution of eventual expressions in parallel. The basic form is the following:
<pre>
“all” “{“
<expression>
“}” ( “and” “{“
<expression>
“}” )+
</pre>
The short form is the following:
<pre>
“all” “(“ <expression> ( “,” <expression> )+ “)”
</pre>
Semantics is the following:
#All expressions starts to be executed.
#If all expressions completed successfully, a list of results is returned.
#If at least one expression fails, an exception that contains generated results and failures is generated.

Example:
<pre>
all {
first()
} and {
second()
}
</pre>
It is translated like the following:
<pre>
{
var wereFaults = false;
var results = makeFlexList();
results.setSize(2)
var faults = makeFlexList();
faults.setSize(2)
def vow1 := seq def r: = {
first()
} then {
results[0] := r
} catch(e) {
wereFaults = true
faults[0] := e
null
}
def vow2 := seq def r: = {
second()
} then {
results[1] := r
} catch(e) {
wereFaults = true
faults[1] := e
null
}
when (def v1:=vow1, def v2:=vow2) {
if(wereFaults) {
throw makeAllFault(results,faults)
} else {
results.toConsList()
}
}
}
</pre>
There is also a loop form that returns conslist of results.
<pre>
“all” “for” <pattern> “in” <collection> “ “{“
<expression>
“}”
</pre>

==Any==
Note: This is most tricky of the proposed operators. And I have not worked out details for E language yet. This construct is much simpler to do right in statically typed class based languages.

This operator allows execution of several eventual expressions in parallel and picking the first available result. The basic form is the following:

<pre>
“any” (“suppress_faults”)? (“compensated”)? “{”
<expression>
“}” ( “or” (“compensated”)? “{”
<expression>
“}” )+
</pre>

The suppress faults modifier affects how faults are treated. If it presents, the faults are returned only if all alternatives returned failed. In this case the first received fault is thrown from any.

The compensated modifier specifies that the branch will return a near value that implements a Compensated interface:

<pre>
interface Compensated {
# this method is called by any operator to produce a value
to run() : Vow[any]
# this method is called by any operator if some branch has
# produced a result. Implementation that does nothing
# is a valid implementation. The method gives an branch
# a chance to cancel production of result.
to cancel()
# If value from run method was produced but not returned
# from any operator, this method is invoked with produced
# value as an argument.
to compensateValue(value : any)
# If run method failed with fault, the resulting fault
# can be compensated as well if it is not used (for example
# logged).
to compensateFailure(fault)
}
</pre>

<pre>
any compensated {
object _ {
to run():Vow[any] {
socketFactory<-connect("localhost", 2222)
}
to cancel() {}
to compensateValue(socket) {
print "eventually connected, but socket is closed"
socket<-close()
}
to compensateFailure(e) {
print e
}
}
} or {
# The method is assumed to throw timeout fault after specified
# amount of milliseconds. Note that might have been a good idea
# to implement cancel operation for timer task. Note that this
# method always finishes with a fault.
timer<-timeoutMs(60000)
}
</pre>

This is translated as the following:
<pre>
later {
def [promise, resolver] := makePromise()
var branch1_finished = false
var branch2_finished = false
var c1 = object _ {
to run():Vow[any] {
socketFactory<-connect("localhost", 2222)
}
to cancel() {}
to compensateValue(socket) {
print "eventually connected, but socket is closed"
socket<-close()
}
to compensateFailure(e) {
print e
}
}
# run first branch
seq def value := {
c1.run()
} then {
branch1_finished := true
if(!branch2_finished) {
resolver<-resolve(value)
} else {
try {
c.compensateValue(value)
} catch(e) {log(e)}
}
} catch(e) {
branch1_finished := true
if(!branch2_finished) {
resolver<-smash(e)
} else {
try {
c.compensateFault(value)
} catch(e) {log(e)}
}
}
# run second branch
seq def value := {
timer<-timeout(60000)
} then {
# a dead branch in our case
branch2_finished := true
if(!branch1_finished) {
resolver<-resolve(value)
try {
c1.cancel()
} catch(e) {log(e)}
}
} catch(e) {
branch2_finished := true
if(!branch1_finished) {
resolver<-smash(e)
try {
c1.cancel()
} catch(e2) {log(e2)}
}
}
promise
}
</pre>

There is also a loop form that returns the first result.
<pre>
“any” (“suppress_faults”)? “for” <pattern> “in” <collection> (“compensated”)? “{”
<expression>
“}”
</pre>

Proposed Asynchronous Control Flow Operators

Const — Tue, 20 Nov 2007 18:13:57 GMT

Const:

Proposed Asynchronous Control Flow Operators

Const — Tue, 20 Nov 2007 17:49:00 GMT

Const: Initial submission

Future research topics

Const — Tue, 20 Nov 2007 17:35:03 GMT

Const: Added entry on stalled question of possible control flow enhancements (the proposal text will follow shortly to the link)

*Develop a translator from E to some formalism that's good for reasoning about authority, such as SCOLL or perhaps an appropriate process algebra. This would have obvious connections to language design and translation, as well as some mathematical / formal analysis leanings. One might even be able to consider this in the context of proof carrying code -- E modules carrying proofs about their security properties would be an interesting thing indeed.

*Investigate the connection between E and the work on using type systems to enforce capability security ala "lightweight static capabilities".

*Take up the CapDesk mantle and build a robust POLA desktop on top of E, using available tools such as Plash or other permission-limiting application environments in order to allow the integration of legacy applications. This would likely also include looking at how existing application launching and packaging frameworks can be leveraged (such as the XML .desktop files used by GNOME for example)

*Complete the E-Native work, which would result in an E runtime environment that isn't layered on top of another runtime environment (e.g., JR, Lisp, Squeak). Unique Research Contributions:
**Develop a VM (OCVM?) that embodies object capability semantics
**Implement the OCVM in a verifiable language (e.g., BitC)
**Develop the Kernel-E AST to OCVM compiler
**Develop a (verifiable?) native implementation of CapTP (although the current implementation of CapTP in Java will probably be replaced soon with one written in E)

*Re-implement COUGAAR [http://www.cougaar.org/] in E to show how object-capability security and promises can be used to simplify and secure distributed, concurrent agent planning systems. Unique Research Contributions:
**Intrinsically secure distributed agent system
**Early (first?) large-scale E (object capability) application
**Worked example of applying object capability patterns to the design of a large distributed system

*EBASS (E-Based Application Server System). Develop an E application server framework (like J2EE or the CORBA Component Model) that provides an infrastructure for deploying general-purpose E services, while orthogonally managing and enforcing organization security policies using object capability patterns. Unique Research Contributions:
**Intrinsically secure application server system
**Early (first?) large-scale E (object capability) application
**Worked example of applying object capability patterns to the design of a large distributed system

*Invent a way to securely re-use native code libraries in E applications or perhaps Emily. Details can be found in [http://www.eros-os.org/pipermail/e-lang/2007-April/011985.html], [http://www.eros-os.org/pipermail/e-lang/2007-April/011987.html], [http://www.eros-os.org/pipermail/e-lang/2007-April/011988.html] and [http://www.eros-os.org/pipermail/e-lang/2007-April/011989.html].

*Implement a version of Second Life that works in a more distributed manner. Instead of relying on just centralized servers, spread the processing and storage of objects to all the end-user machines instead.

*Implement a content management system - like ZOPE - in E. This would be especially true if E moves to using Tyler Close's HTTPSY protocol beneath CapTP, since it would then be directly usable by standard web browsers.

*Expanding CapDesk to become a generalized secure rich client platform (CapClient?) would be a tremendous development. Combined with EBASS, we would then have the ability to create end-to-end secure applications.

*Auditing [http://www.erights.org/elang/kernel/auditors/]. Note that the E-on-CL auditing system is quite different from the one documented here, with interesting advantages and disadvantages. The current E-on-Java is slowly working towards yet another take on an auditing architecture, but it's much less far along. Joe-E special cases some built-in auditors, but we've talked about generalizing it to an openly extensible static auditing framework. During the Waterken security review, we found that around 25% of the classes in the Waterken system were Powerless. In other words, they had passed the Joe-E static checker assuring that they were authority-free. As a result, we were able to ignore these regarding many security-relevant questions during the review. This demonstrates that the auditing idea has a lot of value.

*Safe Serialization Under Mutual Suspicion [http://erights.org/data/serial/jhu-paper/]. The issues involved in serializing a graph of mutually suspicious objects is otherwise unexplored. At these pages, I think we raise more questions than we answer. The need for a good serialization framework meeting these goals seems to come up over and over again in different contexts. The notion (due to Jonathan Rees) of serialization as un-evaluations seems like a very rich starting point. In a recent conversation with Alan Karp and Marc Stiegler, we arrived at a new perspective on un-evaluation. At one simple extreme, we have the literal realism of orthogonal persistence, where objects simply get to keep whatever rights they acquired. We assume that a held permission can only be acquired "legitimately", so the holding of a permission is an adequate demonstration of a the right to continue holding that permission. At a hypothetical simple opposite extreme approximated by SPKI, the entire delegation chain by which a permission was actually acquired is recorded, and is presented afresh each time the permission is exercised, effectively as a chain of title (sharings, not transfers), enabling a fresh check each time that the permission was acquired legitimately. When these two give extremes always give the same answer, the difference is uninteresting. However, if the "rules" change, the first "historyless" system can only grandfather-in all permissions acquired so far by the old rules. OTOH, the SPKI-like system can determine whether to accept as legitimate, under the new rules, the means by which the permissions were acquired under the old. (Note: in general, and especially in the human world, I generally prefer historyless rights systems over retro-active re-evaluations. But both have a place.) Un-evaluation is a complex expressive mid-point between these two. In order for the object to retain its rights across a serialization boundary, i.e., to reconstitute an object like itself with permissions like those it currently holds, it must be able to construct from its current state a rationalization (the uncall), i.e., a simplified story about how it could have gotten these permissions. In the world in which this expression is evaluated, it may or may not succeed at re-obtaining permissions like those it held.

*Decision Alignment (see [http://www.eros-os.org/pipermail/e-lang/2007-April/011997.html]).

*Solve upgrade and legacy compatibility issues (see [http://www.eros-os.org/pipermail/e-lang/2007-April/011998.html]).

*Continue the work of Alfred Spiessens on formal systems for reasoning about authority propagation and practical tools using them. This work would continue his thesis work: [http://www.info.ucl.ac.be/people/PVR/fsp_thesis.pdf].

*Alfred Spiessens' point 2 on page 282 of his thesis.

*Investigate possible enhancement to set of E asynchronous control flow operators [[Proposed Asynchronous Control Flow Operators]]