Modules and export mechanisms

Developer
Oct 12, 2012 at 3:04 AM

General proposal

I like distributed systems and our module system should be of distributed nature as well.

I propose we use something existing for distributing responsibilities: the domain name system. In essence, it splits the whole world into namespaces that themselves can be splitted again. If we use that concept we can use the whole infrastructure for free (zone administration, name servers etc.).

To give you a taste of how I imagine it to be:

Currently we have one namespace that is mapped to the filesystem:

 

import system.core.date

 

I would rather like to have something like the following:

 

import .com.microsoft.koka.sytem.core.date

 

Note that I don't flip the order as it is mostly done, but I keep it top -> bottom equals left -> right as it was meant to be. I also don't omit the root-dot at the beginning. I keep it to make clear that this is an absolute location (more on that later).

At this point I say nothing about where to actually find the modules (filesystem, network, etc.). The koka language definition and semantics should be agnostic about this. All we say is: This is a location in a hierarchiecal ordered namespace and the responsibilities are distributed according to the existing domain name system (I hereby state the assumption that it will be maintained as long as koka exists).

Unqualified and qualified imports

I propose an import mechanism that is easier to understand that Haskell's and equally powerfull. These are the rules:

  1. Each module file creates it's own scope `S`. A scope is flat and just consists of two sets: `modules` and `elements`. `modules` are nodes in DNS while `elements` are the things exported by `modules` like functions, values, types etc. This distinction avoids that modules are more interwoven with the language than necessary.
  2. import .com.example.foo
    
     adds the exports of .com.example.foo to S. This is called an `unqualified import`. Names don't clash until they are actually used (due to koka's type directed name resolution we can resolve most clashes).
  3. import .com.example.foo as .foo
    
    binds directly under S by the name foo. This is called a `qualified import`. In contrast to Haskell the contents of foo are only available in that subnamespace. This behavior is equal to
    import qualified .com.example.foo as .foo
    
    in Haskell. To have foo available at top level and bound to the name foo you simply state
    import .com.example.foo
    import .com.example.foo as .foo

Referencing

Assume the element x is in S (the module internal namespace). To reference x we just say

x

I hereby propose that for referencing elements within an explicit namespace we always prepend a dot, since when we decided to have capital constructor names, things got a bit more ambigious. Furthermore, the DNS is case insensitive anyway (more on allowed characters later).

Namespaces must always be announced (imported) at the beginning of the module (either unqualified or qualified). Assuming

import .com.example.foo as bar

 

we reference x in .com.example.foo by

.bar.x

 

I short: Without the dot, we reference elements; with dot, we reference modules. Easy, isn't it?

Local imports

I agree that it is tedious to write first and second level domains all the time when importing something.

import foo

is just shorter. So we just allow this (qualified and unqualified) and say it is semantic sugar for

import .localhost.foo

We call this a `local (qualified/unqualified) import`.

Versioned imports

I want to at least keep the opportunity to have versioning support in the language itself. Not being able to use two different versions of one and the same library in Haskell taught me, that we might want to have that. I propose a simple linear versioning: each subnamespace can be versioned and that version is referenced to by a natural number.

Syntax:

import .localhost.system[42].core

Again, we say nothing about the actual implementation of a versioning (could be numbered folders in a filesystem, could be Mercurial, whatsoever).

We might stipulate that a version of a module must not be characterwise equal for a lifetime, but program semantics must be the same. This would allow for security and performance fixes, but still guarantee buildability (this is the reason why I chose natural numbers for version indexing and not cryptographic checksums like for example git does. Objections on that?).

Mapping namespaces

It's the compiler's job to give an interpretation for names and modules.

We should proceed in the following order:

  1. We can start by supporting local imports only and keep our current mapping to the filesystem for that. Versioning is also totally optional an can be omitted by now. Probably, we always want to ship the base libraries with the compiler as a filesystem tree which is totally in agreement with what I have in mind.
  2. To implement foreign namespaces, they could be served via plain http on a specific port and the compiler calls a script to download and cache them.
  3. One more thing I have in mind are local aliases: Instead of referencing foreign namespaces all around your module files, you might want to tell the compiler that `localhost.ms` or short `ms` is an alias for `.com.microsoft.koka`  when compiling. This allows you to easiliy change vendor or even pull that module from someone and make local changes.
  4. We might still want to have a central instance that for example maintains the base libraries. Imagine someone develops a new library under his own namespace that is overly useful and we decide to include it in base. We now want to have it under our namespace. No problem: The original developer orders his http server to serve http redirects to us when asked for that module now. No old code is broken and the compiler/build tool could say: `old location is deprecated. Module is now maintained by xy.`. 
  5. The server could serve from a mercurial repository and serve different versions of the same file (also very easy).

This is just an example of how it might be, but I think it would be easy to do and would solve the problem of centralism without damning central instances.

Exports

I propose qualified and unqualified exports with similar semantics like imports.

Remember the module internal scope S. We can say that S becomes S' by filtering out all private elements from the elements set and setting the module set to the empty set.

We support to ways of modifying S': An unqualified export

export foobar

which add the elements of foobar to the elements set of S' and a qualified export

export foo as bar

which adds foo to the module set of S' by the name bar.

S' is then what is seen from that module from the outside.