Add more to intro.md

doc/intro.md

@@ -4,7 +4,7 @@ Dst is a dynamic, lightweight programming language with strong functional
 capabilities as well as support for imperative programming. It to be used
 for short lived scripts as well as for building real programs. It can also
 be extended with native code (C modules) for better performance and interfacing with
-existing software. Dst takes ideas from Lua, Scheme, Clojure, Smalltalk, and 
+existing software. Dst takes ideas from Lua, Scheme, Racket, Clojure, Smalltalk, Erlang, and 
 a whole bunch of other dynamic languages.
 
 # Hello, world!
@@ -39,7 +39,8 @@ in the rest of the tuple.
 
 # A bit more - Arithmetic
 
-Any programming language will have some way to do arithmetic. 
+Any programming language will have some way to do arithmetic. Dst is no exception,
+and supports the basic arithemtic operators
 
 ```
 # Prints 13
@@ -52,7 +53,7 @@ prefix notation. Dst also supports the modulo operator, or `%`, which returns
 the remainder of integer division. For example, `(% 10 3)` is 1, and `(% 10.5 3)` is
 1.5. The lines that begin with `#` are comments.
 
-Dst actually has to flavors of numbers; integers and real numbers. Integers are any
+Dst actually has two "flavors" of numbers; integers and real numbers. Integers are any
 integer value between -2,147,483,648 and 2,147,483,647 (32 bit signed integer). 
 Reals are real numbers, and are represented by IEEE-754 double precision floating point
 numbers. That means that they can represent any number an integer can represent, as well
@@ -78,6 +79,90 @@ Numbers can also be in scientific notation such as `3e10`. A custom radix can be
 as for scientific notation numbers, (the exponent will share the radix). For numbers in scientific
 notation with a radix besides 10, use the `&` symbol to indicate the exponent rather then `e`.
 
+## Arithmetic Functions
+
+Besides the 5 main arithmetic functions, dst also supports a number of math functions
+taken from the C libary `<math.h>`, as well as bitwise operators that behave like they
+do in C or Java.
+
+# Strings, Keywords and Symbols
+
+Dst supports several varieties of types that can be used as labels for things in
+your program. The most useful type for this purpose is the keyword type. A keyword
+begins with a semicolon, and then contains 0 or more alphanumeric or a few other common
+characters. For example, `:hello`, `:my-name`, `:=`, and `:ABC123_-*&^%$` are all keywords.
+Keywords are actually just special cases of symbols, which are similar but don't start with
+a semicolon. The difference between symbols and keywords is that keywords evaluate to themselves, while
+symbols evaluate to whatever they are bound to. To have a symbol evaluate to itself, it must be
+quoted.
+
+```lisp
+# Evaluates to :monday
+:monday
+
+# Will throw a compile error as monday is not defined
+monday
+
+# Quote it - evaluates to the symbol monday
+'monday
+
+# Or first define monday
+(def monday "It is monday")
+
+# Now the evaluation should work - monday evaluates to "It is monday"
+monday
+```
+
+The most common thing to do with a keyword is to check it for equality or use it as a key into
+a table or struct. Note that symbols, keywords and strings are all immutable. Besides making your
+code easier to reason about, it allows for many optimizations involving these types.
+
+```lisp
+# Prints true
+(= :hello :hello) 
+
+# Prints false, everything in dst is case sensitive
+(= :hello :HeLlO) 
+
+# Look up into a table - evaluates to 25
+(get {
+    :name "John"
+    :age 25
+    :occupation "plumber"
+} :age)
+```
+
+Strings can be used similarly to keywords, but there primary usage is for defining either text
+or arbitrary sequences of bytes. Strings (and symbols) in dst are what is sometimes known as 
+"8-bit clean"; they can hold any number of bytes, and are completely unaware of things like character
+encodings. This is completely compatible with ASCII and UTF-8, two of the most common character
+encodings. By being encoding agnostic, dst strings can be very simple, fast, and useful for 
+for other uses besides holding text.
+
+Literal text can be entered inside quotes, as we have seen above.
+
+```
+"Hello, this is a string."
+
+# We can also add escape characters for newlines, double quotes, backslash, tabs, etc.
+"Hello\nThis is on line two\n\tThis is indented\n"
+
+# For long strings where you don't want to type a lot of escape characters,
+# you can use two backslashes with 0 or more equal signs inside them.
+# To close this string, simply repeat the opening sequence (with a matching number of = characters).
+\====\
+This is a string.
+Line 2
+    Indented
+"We can just type quotes here", no problem.
+\====\
+
+# You don't need any = charcters in the delimiters
+\\
+This works if two backslashes don't appear in your string.
+\\
+```
+
 # Functions
 
 Dst is a functional language - that means that one of the basic building blocks of your
@@ -87,7 +172,7 @@ created as needed.
 
 Functions can be defined with the `defn` macro, like so:
 
-```
+```lisp
 (defn triangle-area [base height]
  (print "calculating area of a triangle...")
  (* base height 0.5))
@@ -102,7 +187,7 @@ form in the body is what the function evaluates to, or returns.
 Once a function like the above one is defined, the programmer can use the `triangle-area`
 function just like any other, say `print` or `+`.
 
-```
+```lisp
 # Prints "calculating area of a triangle..." and then "25"
 (print (triangle-area 5 10))
 ```
@@ -118,7 +203,7 @@ as arguments to other functions as well
 (print triangle-area)
 ```
 
-Prints the location in memory of the function triangle area. This idea can be used
+This prints the location in memory of the function triangle area. This idea can be used
 to build some powerful constructs purely out of functions, or closures as they are known
 in many contexts.
 
@@ -134,6 +219,25 @@ The above expression first creates an anonymous function that adds twice
 the first argument to the second, and then calls that function with arguments 10 and 20.
 This will return (10 + 10 + 20) = 40.
 
+There is a common macro `defn` that can be used for creating functions and immediately binding
+them to a name. `defn` works as expected at both the top level and inside another form. There is also
+the corresponding
+
+```lisp
+(defn myfun [x y]
+ (+ x x y))
+
+# You can think of defn as a shorthand for def and fn together
+(def myfun-same (fn [x y]
+            (+ x x Y)))
+
+(myfun 3 4) # -> 10
+```
+
+Dst has many macros provided for you (and you can write your own).
+Macros are just functions that take your source code
+and transform it into some other source code, usually automating some repetitive pattern for you.
+
 # Defs and Vars
 
 Values can be bound to symbols for later use using the keyword `def`. Using undefined
@@ -157,3 +261,200 @@ a var.
 (:= myvar 10)
 (print myvar)
 ```
+
+In the global scope, you can use the `:private` option on a def or var to prevent it from
+being exported to code that imports your current module. You can also add documentation to
+a function by passing a string the def or var command.
+
+```lisp
+(def mydef :private "This will have priavte scope. My doc here." 123)
+(var myvar "docstring here" 321)
+```
+
+## Scopes
+
+Defs and vars (collectively known as bindings) live inside what is called a scope. A scope is
+simply where the bindings are valid. If a binding is referenced outside of its scope, the compiler
+will throw an error. Scopes are useful for organizing your bindings and my extension your programs.
+There are two main ways to create a scope in Dst.
+
+The first is to use the `do` special form. `do` executes a series of statements in a scope
+and evaluates to the last statement. Bindings create inside the form do not escape outside
+of its scope.
+
+```lisp
+(def a :outera)
+
+(do
+ (def a 1)
+ (def b 2)
+ (def c 3)
+ (+ a b c)) # -> 6
+
+a # -> :outera
+b # -> compile error: "unknown symbol \"b\""
+c # -> compile error: "unknown symbol \"c\""
+```
+
+Any attempt to reference the bindings from the do form after it has finished
+executing will fail. Also notice who defining `a` inside the do form did not
+overwrite the original definition of `a` for the global scope.
+
+The second way to create a scope is to create a closure.
+The `fn` special form also introduces a scope just like
+the `do` special form.
+
+There is another built in macro, `let`, that does multiple defs at once, and then introduces a scope.
+`let` is a wrapper around a combination of defs and dos, and is the most "functional" way of
+creating bindings.
+
+```lisp
+(let [a 1
+      b 2
+      c 3]
+      (+ a b c)) # -> 6
+```
+
+The above is equivalent to the example using `do` and `def`.
+This is the preferable form in most cases,
+but using do with multiple defs is fine as well.
+
+# Data Structures
+
+Once you have a handle on functions and the primitive value types, you may be wondering how
+to work with collections of things. Dst has a small number of core data structure types
+that are very versatile. Tables, Structs, Arrays, Tuples, Strings, and Buffers, are the 6 main
+built in data structure types. These data structures can be arranged in a useful table describing
+there relationship to each other.
+
+|            | Mutable | Immutable       |
+| ---------- | ------- | --------------- |
+| Indexed    | Array   | Tuple           |
+| Dictionary | Table   | Struct          |
+| Byteseq    | Buffer  | String (Symbol) |
+
+Indexed types are linear lists of elements than can be accessed in constant time with an integer index.
+Indexed types are backed by a single chunk of memory for fast access, and are indexed from 0 as in C.
+Dictionary types associate keys with values. The difference between dictionaries and indexed types
+is that dictionaries are not limited to integer keys. They are backed by a hashtable and also offer
+constant time lookup (and insertion for the mutable case).
+Finally, the 'byteseq' abstraction is any type that contains a sequence of bytes. A byteseq associates
+integer keys (the indices) with integer values between 0 and 255 (the byte values). In this way,
+they behave much like Arrays and Tuples. However, one cannot put non integer values into a byteseq.
+
+```lisp
+(def mytuple (tuple 1 2 3))
+
+(def myarray @(1 2 3))
+(def myarray (array 1 2 3))
+
+(def mystruct {
+ :key "value"
+ :key2 "another"
+ 1 2
+ 4 3})
+
+(def another-struct
+ (struct :a 1 :b 2))
+
+(def my-table @{
+ :a :b
+ :c :d
+ :A :qwerty})
+(def another-table
+ (table 1 2 3 4))
+
+(def my-buffer @"thisismutable")
+(def my-buffer2 @\====\
+ This is also mutable ":)"
+ \====\)
+```
+
+To read the values in a data structure, use the get function. The first parameter is the data structure
+itself, and the second parameter is the key.
+
+```lisp
+(get @{:a 1} :a) # -> 1
+(get {:a 1} :a) # -> 1
+(get @[:a :b :c] 2) # -> :c
+(get (tuple "a" "b" "c") 1) # -> "a"
+(get @"hello, world" 1) # -> 101
+(get "hello, world" 0) # -> 104
+```
+To update a mutable data structure, use the `put` function. It takes 3 arguments, the data structure,
+the key, and the value, and returns the data structure. The allowed types keys and values
+depend on what data structure is passed in.
+
+```lisp
+(put @[] 100 :a)
+(put @{} :key "value")
+(put @"" 100 92)
+```
+
+Note that for Arrays and Buffers, putting an index that is outside the length of the data structure
+will extend the data structure and fill it with nils in the case of the Array, 
+or 0s in the case of the Buffer.
+
+The last generic function for all data structures is the `length` function. This returns the number of
+values in a data structure (the number of keys in a dictionary type).
+
+# Flow Control
+
+:)
+
+# Combinators
+
+:)
+
+# Modules
+
+:)
+
+# The Core Library
+
+Dst has a built in core library of over 200 functions and macros at the time of writing.
+While some of these functions may be refactored into separate modules, it is useful to get to know
+the core to avoid rewriting provided functions.
+
+For any given function, use the `doc` macro to view the documentation for it in the repl.
+
+```lisp
+(doc defn) -> Prints the documentation for "defn"
+```
+To see a list of all global functions in the repl, type the command
+
+```lisp
+(getproto *env*)
+```
+Which will print out every built-in global binding
+(it will not show your global bindings). To print all
+of your global bindings, just use *env*, which is a var
+that is bound to the current environment.
+
+# Prototypes
+
+:)
+
+# Fibers
+
+:)
+
+# Macros
+
+:)
+
+# IO
+
+:)
+
+# The Parser Library
+
+:)
+
+# The Assembler
+
+:)
+
+# Interfacing with C
+
+:)