Thoughts on Lazy Evaluation...

... I'll do it later.

Stuff...

So I got an iPad. I've had it about one month. I've been paying a lot of attention to the talk about how it doesn't do Flash, why it doesn't do Flash, how Apple is committing war crimes against humanity by disallowing applications authored in 3rd party tools etc etc.

As a developer who's spent a good bit of time working on different projects that scale from tiny little machines, to medium sized computers to giant supercomputing clusters (yes, I've been on several of the top 10 of the top500 list, writing software to squeeze performance out of them) I can tell you that flexible tool chains, great documentation, and great support do not always go hand in hand.

My opinion on that is that it's a bit sad that I won't be writing and running Haskell code on a non-jailbroken i(Phone|Pod|Pad) but that that's not a deal breaker for most people. Cocoa is a nice framework, with many years behind it making it great. Objective-C is a pretty cool language, (though I feel they should have kept it simpler, no garbage collection, all this automatic atomic update stuff can be confusing etc). Grand Central Dispatch and the libdispatch stuff is powerful, even in a raw C programming context, though some folks I know don't think it's well served to use it outside the realm of Objective-C. Having suffered programming with threads and locks, (even implementing my own locks on certain platforms) I'd say that this is a big step forward in thinking about concurrency and parallelism by means of organizing program code at a low level.

Yes, when you buy Apple's stuff, it's a bit more about doing things "their way" than doing things "your way". The limits Apple places on the hardware it supports with its operating systems, or the limits placed on programmers via the tool chains, are all really there for 2 reasons (in my opinion).

1. It keeps Apple from losing control of its own platform.

2. Apple can focus their engineering and support efforts on making a product that seemingly "just works" with all supported stuff, because the space of stuff to support is a lot smaller!

To me, neither of these things are inherently "evil", as some folks might like to convince you.

I should note I've used my iPad every day since I got it at least one time/day. To pay for it I began selling some of the stuff I intend not to need anymore, such as my old iPod touch, and my old laptop. So far it's been a great trade!

Just a thought or two on strictness

I've noticed that a lot of times when it comes to profiling programs in any language, that it is often surprising where the big cost centers are. Maybe it's just due to experience, but a lot of times I at least feel like I'm not as surprised in imperative languages when I recognize a cost center or my tool chain does. With functional languages, I'm often surprised that certain implementations of functions are as costly as they are, while others are not. This is even more true in "lazy" functional languages like Haskell.

When you're dealing with strict evaluation by default, you can basically just read the program top down and get a good idea for what happens when, and how data much a particular chunk of code ought to be using, and for the most part you'll get this right with experience.

With a language like Haskell, sometimes, all bets are off unless you know very well how a particular library of functionality is implemented. There's both strict and non-strict versions of Control.Monad.State for example. The non-strict version almost requires you to read the code, then think about how much stuff can be deferred to the last minute, and realize that those little deferred computations will have to be evaluated later. This can not only change the order of evaluation and thusly the data size of your program, but can also change the behavior a great deal.

Data.Map has operations within it for update that will be deferred (non-strictly evaluated) and some that are strict. It's important to read the documentation.

My advice to beginners is to try to understand strictness vs non-strictness (often called laziness) in Haskell as early as possible. It will give you great leverage to expressing some really beautiful solutions to problems, and save you from pulling all of your hair out or suffering any existential crises when trying to figure out why a program doesn't do what you thought.

All of that said, it's pretty easy for me to see why some people will decide that non-strict functional programming languages are not for them. It certainly takes a little mind bending to see the value in it if all you've ever done is imperative or strict functional programming, but I promise you the value is truly there. I also find that by challenging myself to learn how to think in these new paradigms I am simultaneously strengthening my understanding of the older paradigms a good deal more, so give it a shot and don't give up! There's help out there for you.

NineP is on hackage!

I'm excited about it. There's still work to be done, but now I think there's a good chance to get a lot more feedback about the module as it's in experimental, and hopefully we can get some of these folks interested in distributed systems to use this nice, simple protocol.

9ph package for cabal/hackage

The 9ph code repository now has a fully cabalized module, and I'm waiting on a response to get permission to upload stuff to Hackage. This will be my first shared contribution to the Haskell community, though all I've done is create a simple test program and some of the administrative work around wrapping this code up. If anyone likes this, the credit should go to Tim.

I've got some ideas for enhancements, and ways to make a nice server/client API on top of this lovely encoding/decoding library that Tim created on top of the very excellent binary package for Haskell and the lovely Applicative module.

9ph works!

Finally, since last September, I've gotten around to messing with 9ph again. That's the 9P2000 implementation in Haskell that Tim Newsham basically wrote and sent to me to play around with.

I had started such a project on my own at one point, but Tim totally beat me to it, and rather than just re-implement the wheel, I figured I'd check out what he'd done and checked it in.

Today, I wrote a function that runs an IO program yielding a socket that's connected to a local 9P2000 server (or error'd out actually) running on 127.0.0.1:6872. From there I was able to create messages for Tattach, Topen, and Tread, and could explore the errors and successes of the server I was talking to.

For my experiments I just used the Inferno operating system with

"styxlisten -A 'tcp!*!6872' export /"

This just exports the local / namespace that the process running the styxlisten command can see to the world on TCP protocols listening on port 6872.

This library should be enough to write scripts for what is quickly becoming my favorite X11 window manager (if you must use one) wmii. wmii actually exposes a bit of functionality via a 9P server that you can mount from v9fs, Inferno, or any other 9P client. It'd be nice to be able to write some Haskell to configure wmii I figured, so this might be my first use for this library.

I'm hoping to get back to this again before long. This was good progress. I'm going to check in on Tim and see if he's got any more updates he feels he'd like to push. If not I'll just work from this fork.

Iteratee

Iteratee really looks promising on paper, and people using it seem to think it's really great. I've been put off a bit by what looks like a rather complex interface, but decided last night to take a crack at it.

What I've written below is an iteratee wrapper around "cat /etc/passwd" output using runInteractiveCommand.

module Main where

import Control.Monad.Trans

import Data.Iteratee.Base

import Data.Iteratee.Base.StreamChunk (ReadableChunk (..))

import Data.Iteratee.IO.Handle

import System.Process

import System.IO

-- For some reason this signature is wrong, but I'm not sure why...

--handleDriver :: (MonadIO m, ReadableChunk s el) => IterateeG s el m a -> Handle -> m a

handleDriver iter h = do

result <- enumHandle h iter >>= run

liftIO $ hClose h

return result

main :: IO ()

main = do

(_, outp, _, _) <- runInteractiveCommand "/bin/cat /etc/passwd"

handleDriver (stream2list :: IterateeG [] Char IO String) outp >>= putStrLn

handleDriver just runs enumHandle with an iteratee (in my case stream2list) over the Handle, in blocks (not character by character) that are specified by the implementation, returning the result. That result is then printed to stdout by putStrLn.

This is a little bit like interact for Handle in that I could have used something more advanced than stream2list to process the result of "/bin/cat /etc/passwd".

I'm not too excited about the fact that I got the type signature on handleDriver wrong, and I'm also a little bit put off by the type signature on stream2list.

So what's the difference between this approach and an interact-like styled approach? For one iteratees work like the function that's supplied to a fold operation over a collection of data. In this case the collection is being produced dynamically via the enumerator. The iteratees themselves work like little parsers that can be composed in a monadic sense. Errors in IO and termination of a stream get propagated automatically up through the system nicely.

Why is lazy IO then not so great? Look at the signature for interact:

interact :: (String -> String) -> IO ()

Interact is a function that takes a function of String to String and produces IO. This means it, ostensibly consumes all the input on stdin, applies the provided function to convert the whole String to a String, then prints that string to stdout. Now it doesn't have to read the whole of input in one shot, because of lazy evaluation. Strings are [Char], and the list structure in Haskell is non-strict in its construction. It's like pausing the construction of that list to do some processing on it, and going back to it with coroutines, except that the system is doing it behind the scenes.

What happens in interact when an error occurs? How does the pure function of type (String -> String) even know about exceptions in the processing? This is where iteratee is an improvement on traditionally lazy IO.

Let's assume we wanted to write a lazy version of interact for a handle called hInteract.

hInteract :: Handle -> (String -> String) -> IO ()

I believe this function could be used safely as follows:

withFile "/etc/passwd" ReadMode ((flip hInteract) id)

withFile uses bracket internally to ensure that hClose is called on the handle and all seems well. I don't think we necessarily understand how resources get used. Oleg, the father of Iteratee, posts this message a few years back explaining more of the benefits of Iteratee.

However, it seems that there is now a new lazy IO mechanism available that is safe. I've not had any time to check into this, but I plan to in the next coming days.

Having written an Expect-like Monad, I'm interested in the aspects of error handling and precise resource control, because the code I'm writing really needs to be able to run to as close to forever as I can get.