Log in

  Journal   Friends   Calendar   User Info   Memories

Improved Means for Achieving Deteriorated Ends

22nd August, 2012. 8:34 am. Blogging Overhaul

Folks, this WordPress blog will soon disappear. I’ve wanted to get rid of WordPress for a while and half-wrote some blog software back in Spring 2011. Some coworkers had a similar plight so we all agreed to finish our blog engines in a one week sprint. I’m done and will likely switch over in the coming days. My livejournal, for those of you still reading that, will no longer receive crossposts and go dark. The blog will be at http://blog.redlinernotes.com/ and there will be an RSS feed at http://blog.redlinernotes.com/rss.xml. There is no ATOM support at this time but…you know, patches welcome. :) For the curious, the new software is called Coleslaw after a nickname of my favorite poet Milosz. It’s basically a jekyll-alike in Common Lisp. Anyway, see you on the other side.

Make Notes

17th August, 2012. 11:34 pm. A Fun, Maybe Useful Exercise…

A random walk through 30 years of history…

1960s: Lisp, Simula, Algol, our hallowed forefathers

1971: Unix released in Assembly, the advent of time

1972: Unix ported to C or, the advent of Portable/Commodity OSes

1973: ML interpreter in lisp appears, the advent of Typed FP?

1974: Deutsche+Greenblatt, the advent of LispM Dreams, fast implementations

1975: Sussman and Steele, the advent of Scheme

1976: Copyright Act of 76, the advent of Proprietary Software

1977: Apple II, Atari 2600, the advent of ‘hobby computing’

1978: 8086, The first BBS, TCP split into TCP/IP, the advent of networking

1979: CADR LispM paper published, the advent of Single-User Workstations?

1980: Smalltalk-80, the advent of OOP? (76 wasn’t released outside PARC)

1981: Symbolics LM-2 sells, the advent of the LispM

1982: Commodore 64; Sun-1 workstation or, the advent of the Unix Workstation

1983: MSFT announces Windows; Stallman founds GNU, the advent of free software

1984: Apple Macintosh or, the advent of consumer graphical PCs

1985: Symbolics registers first domain, NES, Intel sells 386, the advent of home computing? (spreadsheets, office suites, etc)

1986: IETF is formed, IMAP is developed, efforts to add OO to Common Lisp, more standards

1987: GNU Compiler Collection, the advent of free toolchains

1988: Soundblaster, MS-DOS 4.0, a boring year

1989: General Public License, the arrival of free software

1990: Haskell 1.0, Caml Light, the arrival of Typed FP? not quite yet…

1991: Linux 0.01, the advent of open source

1992: Alpha+OpenGenera, Linux GPL’d, Windows 3.1 is released, the advent of the Dark Ages

1993: Pentium I released, the advent of the “One True”(ly bad) Architecture

1994: Linux 1.0, XFree86, Red Hat, the advent of the distro

1995: IE 1, AOL 3 hits 2 million users, the advent of the net

1996: Linux 2.0 w/SMP, Apache leads web servers, the arrival of open source

1997: Wifi+HTTP 1.1, CD-RWs+Office 97, IE 4+google.com, nullsoft founded, the advent of the modern age

1998: US v MSFT, Netscape open sources Mozilla, the 6-year browser war purgatory

1999: Google Incorporates, the advent of Search

Make Notes

22nd June, 2012. 12:49 pm. An Emulator Design Pattern

Going into this project, I knew almost nothing about emulation. I still know very little. But I was tired of seeing emulators written in C and Java for performance or portability that wound up looking like a big switch statement sooner or later. My 6502 emulator is ANSL CL and should run anywhere that sbcl, clisp, ccl, or any other fine Common Lisp implementation will run. Granted, if a totally new processor architecture comes out, it’s probably easier to cross-compile/port a 6502 in C than the CL compiler hosting cl-6502 but I digress. I wanted to write a fast, high-level emulator. The closest thing I found to an emulator design that I liked was py65 but even that was a bit less abstract than I would’ve hoped.

I’m still searching for ways to improve the core abstractions that I have (and no doubt there are many hardcore low-level hackers that would be disgusted by my work) but I’m enjoying the process and here are some preliminary thoughts…

Addressing modes!


Addressing modes, addressing modes, addressing modes. This is the biggest difference between assembly language and the languages I use day to day. Hence, it’s a crucial abstraction to get correct. For the first time, Common Lisp’s notion of “generalized places” has been a real boon. There were two nuances to addressing modes that I found I really needed to account for.

  • Some modes access CPU registers, others RAM. (This mostly just effects the way I need to use setf.)

  • Most opcodes use the byte /at/ an address rather than the address, but sometimes an opcode *does* need the address.

To solve these issues and abstract some code patterns, I wrote a macro called defaddress. It defines a method on the CPU that returns the address computed by the mode /and/ generates a setf function that sets the register in the CPU struct or the byte in memory based on a cpu-reg keyarg to the macro. Finally, to solve the issue that we most often want the byte, opcodes are defined as either being :raw or not (the default). If they’re not, instead of passing the mode symbol to be funcalled, we pass a lambda that gets the byte at the address computed by the mode. So far, concerns seem nicely separated. Time will tell if I’ve struck on the right design here.



Opcodes in cl-6502 are really mnemonics, a set of opcodes that encode the same language primitive but for different addressing modes. As long as the foundation of addressing modes as funcallables is there to support the opcodes, you can write any opcode cleanly with a single body of code shared across all addressing modes. This has made me deeply happy as it seems to me to be _THE RIGHT THING_.

The implementation of defopcode is a bit hairy, particularly the EVAL-WHEN block to make sure metadata about the opcodes gets set in the *opcodes* array at load-time, but the supporting defins macro is fairly clean. The important thing is that opcode definitions wind up looking marvelous. For example, here’s ASL and BCC:

    (defopcode asl (:docs "Arithmetic Shift Left" :raw t)
        ((#x06 5 2 'zero-page)
         (#x0a 2 1 'accumulator)
         (#x0e 6 3 'absolute)
         (#x16 6 2 'zero-page-x)
         (#x1e 7 3 'absolute-x))
      (update-flags (funcall mode cpu) '(:carry))
      (let ((result (wrap-byte (ash (funcall mode cpu) 1))))
        (update-flags result)
        (funcall setf-form result)))

    (defopcode bcc (:docs "Branch on Carry Clear" :track-pc nil)
        ((#x90 2 2 'relative))
      (branch-if (lambda () (zerop (status-bit :carry cpu))) cpu))

ASL defines 5 methods here each with a different addressing mode but sharing the same body. They update flags in the status register as expected, increment the program counter properly, and put metadata in *opcodes* to aid with dispatch and disassembly. Not bad, eh?

Objects, Functions, Macros, Whatever!


So far, I’ve mostly just used objects (read: CLOS) to define some conditions, two core methods on the CPU (step and execute), and the instructions themselves. The CPU itself is defined as a Struct rather than a Class. All the instructions are methods EQL-specialized on the opcode and the opcode alone which should make dispatch pretty speedy. The methods reference the *cpu* global directly and since Common Lisp has /usable global variables/ that look up the most recent binding in the current thread on reference, I should be able to run many instances safely in different threads on a single core. Just do something like…

    (make-thread :foo (lambda ()
           (let ((*ram* (make-array (expt 2 16) :element-type '(unsigned-byte 8))
                 (*cpu* (make-cpu))))
             &body))) ;; and we're off to the races!

What’s next?


I have a bunch of potential ideas for what’s next. I want to extend this work towards full NES emulation in the browser. There isn’t a formal ordering of priorities or milestones yet (cause this is kind of an art project), but coming up with a sane, RESTful API to sit on top of cl-6502 is probably the next step. Hopefully I’ll get some time to hack on that this weekend.

  • cl-6502: An assembler! Unit tests + bug fixing.

  • famiclon: An NES emulator backend built on romreader and cl-6502. Video, Sound. Input?
  • Qeng-Ho: Hunchentoot+ST-JSON, REST API wrapping cl-6502+famiclon, No persistence! Multiple CPUs! Internal private API for now.

  • Pham-Nuwen: Clojure+Clojurescript@Deepclouds.net. Persistence! Nice web interface. Graphics w/canvas! Etc…

  • cl-z80: DO THAT SHIT! (how hard could it be? just another 8bit thing. see emu-docs.org)

Make Notes

17th June, 2012. 11:27 pm. On Interactive Retrocomputing

Lately, I’ve been working on an emulator for the MOS 6502 processor in Common Lisp that I’ve been boring enough to name cl-6502. The emulator is basically finished as is the disassembler. There are also pretty solid docs. An assembler should be added soon and hopefully a helper utility or two and unit tests. But why do this? Well, for a couple of reasons.

1) I never did enough Systems Programming. I never did any assembly in college and wrote an absolute paucity of C (granted, that’s my fault). I never learned enough about the inner working of Operating Systems or how to exploit memory hierarchies. I want to know more about how the machine works at a low level. Writing an emulator in a high-level language isn’t a great way to do that but I wanted to anyway. Writing some assembly programs to run on this emulator might help though and I hope to do some of that later.

2) I was curious how concise, extensible, and performant an emulator could be written with Common Lisp. Most emulators are written in C/C++ for performance reasons. There are a few in Java (for portability?) or Javascript for what we now call portability but even these are not terribly high-level from a design perspective. I don’t have all the answers to this question yet but I’m excited by some of the work I’ve done so far. In particular, writing macros for Addressing Modes and Opcodes has been quite helpful and I expect CLOS’s :before, :after, and :around methods to go a long way where extensibility is concerned. I’m hoping the EQL-specialized methods, SBCL, and perhaps some shrewd profiling can lead to good performance. Also, 45 lines of code for a 6502 disassembler doesn’t seem bad to me. :P

3) ICU64/Frodo Redpill. I’m not wild about static types and I tend to write tests after the fact because I view programs as clay until they’re ready to be fired in the kiln. Roly Perera’s work on self-explaning computation interests me a lot as does Chris Granger’s work on Light Table. Rapid feedback loops are important. Maintaining flow is important. As far as I’m concerned, all emulators should strive towards the sort of “peek/poke the machine” experience that ICW/Frodo Redpill offers. Games are a very easy way to get people to engage with computers. Everybody likes games. And lots of folks at some point in wondering about programming, ponder how much is involved in changing something about a game they like. With a system like ICU/Frodo Redpill they could literally /see/ the answer. Add an integrated editor and you’re in pretty interesting territory. Feel like changing something about the “hardware”? Feel like having breakpoints and step debuggers pop up on arbitrary memory accesses or instruction executions? You got it. But ICU64/Frodo Redpill has this all locked down on desktops. Why not do as much as possible with HTML5, <canvas>, and Clojurescript? I’m not going to be the guy to come up with the next Mother of all Demos … but I hope to make something cool. And if I’m really lucky, I’ll have something interesting to play with online by Strange Loop on September 23rd. I’ve already got a 6502 emulator. What’s next?

Make Notes

16th June, 2012. 7:30 pm. A tentative Strange Loop 2012 Schedule

Strange Loop has posted their schedule for 2012 and my company has been kind enough to send me. Without further ado, here’s my current thought on which talks I’ll attend. I just can’t wait for September. :)

;; Sunday, September 23 (Emerging Languages Preconf)

– 7:30 flight? ZOMG WHAT WAS I THINKING?!?

09:30 Jeremy Ashkenas – Symbiotic Languages: Transpiling into Javascript

10:30 Ostap Cherkashin – Bandicoot: code reuse for the relational model

11:30 Hakan Raberg – Clever, Classless and Free?

12:40 Michael Fogus – The Reemergence of Datalog

13:20 Brian McKenna – Roy

14:40 David Herman – Rust

15:50 James Noble – Grace: an open source educational OO language

16:30 Jose Valim – Elixir: Modern Programming for the Erlang VM

17:10 David Pollak – Visi: Cultured & Distributed


;; Monday, September 24

09:00 Michael Stonebraker – In-Memory Databases

10:00 Dustin Getz – Monad Examples for normal people, in Python and Clojure

11:00 Pieter Hintjens – Software Architecture using ZeroMQ

– (or Functional Design Patterns – Stuart Sierra)

12:20 Neil Milstead – Augmented Reality and CV

13:00 Craig Kersteins – Postgres Demystified

14:00 Neha Narula – Executing Queries on a Sharded Database

– (or Clojurescript by David Nolen)

15:30 Scott Vokes – Data Structures: The Code That Isn’t There

– (or Lessons from Erlang by Garrett Smith, Types vs Testing by Paul Snively and Amanda Laucher) GAAH

16:30 Rich Hickey – The Database as a Value

17:30 Lars Bak – Pushing the Limits of Web Browsers

20:00 Matthew Taylor – Humanity 2.0

;; Tuesday, September 25

09:00 Jeff Hawkins – Computing Like the Brain

10:00 Chris Granger – Behind the Mirror

11:00 Nathan Marz – Runaway complexity in Big Data…and a plan to stop it

12:20 Carlton Mills – Computer Architecture of the 1960s

13:00 Oleg Kiselyov – Guess lazily! Making a program guess and guess well

14:00 Cliff Moon – The Audubon Society for Partial Failures

15:30 Ola Bini – Expressing Abstraction, Abstracting Expression

– (or Building visual, data-driven UIs with ClojureScript)

16:30 Bret Victor – ?

– And then I’m out because of my 7:30 flight. Sorry Brendan Eich!


Make Notes

6th June, 2012. 1:23 pm. 10 Great Hacking Albums

My posts on this blog have leaned away from technical content for the last year, tending towards the introspective and music or poetry. I’m hoping to start shifting back in the other direction for a little bit and plan to write about my latest personal hacking project soon. In the interim, here are 10 favorite albums to write code to off the top of my head. I will note at the outset that I prefer ambient and instrumental music for hacking. Ambient stuff in particular seems to naturally encourage a state of “flow” for me. Also, here’s a link to some C2 wiki discussion on flow as it relates to programming. Also, as long as we’re throwing out great hacking music I might as well shill the mixtapes I’ve been working on the last two months. :) All three are pretty solid. And there’s a one hour extended mix that mashes together I/Omega and Lost Without a Traceback that isn’t on soundcloud. Ping me if you’re interested. There are some track changeups and much improved transitions in the I./Omega half.


Tim Hecker – Harmony in Ultraviolet

Tim Hecker – An Imaginary Country

Tim Hecker – Ravedeath, 1972

Fennesz – Venice

Fennesz – Black Sea

Fuck Buttons – Tarot Sport

Rustie – Glass Swords

Araabmuzik – Electronic Dream

Four Tet – There Is Love In You

Amon Tobin – Supermodified

(Honorable Mentions: Tycho – Dive, Washed Out – Life of Leisure)

If you were to proceed through the above albums in sequence, you’d start with some fantastic downtempo ambient/noise stuff with Hecker and Fennesz, shift into uptempo “noisetronica” with the Fuck Buttons, transition into the over the top and in your face Rustie, ride those dancey vocals and synths into Araabmuzik, and then start winding down with the more midtempo vocals of There Is Love In You and the outstanding groove of Supermodified.

Make Notes

18th May, 2012. 8:00 am. 3 Years In

It’s been 3 years since Dad died, 2 years since I’ve been single. It’s hard to believe, really. And wouldn’t you know, I had a dream about Dad for the first time in a while. He had cancer but was still alive. We’d all moved back to Cross Creek where I’d spent the first decade of my life or so. I had my job at CMG, I think, and I was talking to Dad who confided in me that he and Mom were having serious financial trouble. I remember being shocked that something hadn’t been mentioned to me sooner and feeling guilty for somehow missing that I could have helped for so long. I’d just been enjoying life as I am these days, a good job, good friends, etc. I was talking to Mom or Dad about how I could support them when I woke up. In the dream, Dad still had all his hair. I don’t think I’ve ever had a dream about him post-hair and post-chemo. I’m thankful for that. He wasn’t the same person once his hair was gone.

I’d been a bit depressed the first half of this week and I couldn’t figure out why. It wasn’t until talking to mom Tuesday night that she reminded me we were coming up on the 3-year anniversary. I’d thought about it when someone asked about dad a few weeks back but it slipped my mind. I’m thankful for my job and coworkers, I love them dearly and even on my grumpiest, most “Office Space-y” days I’m happy to be in the office. I’m thankful for my friends who keep Atlanta fun and keep life interesting even when we all just want to have a drink and be together at the end of a long day. I’m thankful for my mother, who’s weathered plenty and still figuring out how to have fun. Mostly, of course, I’m thankful for dad. I still don’t know how he did it all. But thanks to him, I can too.

Read 1 Note -Make Notes

15th April, 2012. 4:21 pm. April Adventures

I’ve been working on some of the depression issues mentioned a few posts back, making progress. Been making a concerted effort to have fun. Lots of weekend trips to Athens, socializing, concerts, that sort of thing.  Starting to create things again too! I’ve made a mixtape for the first time….like a proper mixtape. Did it in Audacity. Didn’t try to do tempo/beat-shifting or anything fancy, just 29 minutes of cutting and fading. I still think it turned out quite well for a first effort. It leans strongly in the electronic/dance direction so if that’s a big turn off for you steer clear. Anyway, onwards and upwards. And to steal from Milosz and Neruda: “Yes, this is my gift to you. Above ashes on a bitter, bitter earth. To you, to the one who unknowingly has awaited me, I belong and acknowledge and sing.”

Here’s the mix and the tracklist. (EDIT: Here’s a vastly improved version of the mix, the associated tracklist, and where you can listen online.)

Read 1 Note -Make Notes

17th March, 2012. 3:44 pm. A Quiet Saturday

I need another dose of Milosz. Perhaps you do too. Here we go…

Guardian Angel

In my dreams my guardian angel takes the form of a woman,

Not always the same one. He knows that I, a fleshly creature,

Need a lover’s touch. We don’t make love,

But there is a closeness between us, and understanding.

I never believed in the presence of angels, but my dreams have changed,

And when, recently, I found an underground grotto filled with treasure,

And we were moving the sacks together, I asked him

For one more moment of the dream, which gave me peace.

Make Notes

8th March, 2012. 11:39 pm. Cool Waters Ahead

Disclaimer: This is going to be a much more personal entry than normal. Just to get down some recent thoughts and revelations that I’ve had. I’ve read people talking about blogs as subject-focused, about building them almost like products. But this thing is a record, a mosaic, and a journal. So here we go.

On and off depression is the one thing I still really feel like is holding me back and don’t like about myself. Part of the reason I stayed single (even became single in the first place) was to address this issue. I’ve resisted taking medication since I went cold turkey in February of 2005. For the seven years prior to that, I had been on a daily cocktail of Tegretol (a mood stabilizer), Risperdal (an anti-psychotic), Effexor XR (an anti-depressant) and Adderall (ADHD med/legalized extended release cocaine). It was a non-trivial cocktail. I may take up a light antidepressant until I can learn to keep myself in the upswing without it. But this post is less about that and more about celebrating continuing to become mentally and emotionally healthier and making it as far as I have.

Let’s start with that last part. The meds I used to be on were *not* an accident and though I didn’t love the experience they had their use. I demonstrated many of the precursors for Bipolar Disorder and the primary reason I was on the meds was to curb the likelihood of my brain ingraining the sort of patterns that can only be fought by regular Lithium intake later. I have a close blood relative who has suffered from Schizophrenia and another who received 33 shock treatments over the course of a bad episode. I think we can safely say that things have panned out so far. I’ve been medication free for 7 years and all indications are that the meds steered me away from forming some pretty deep canyons for the chemicals in my brain to run through.

In addition to that, and please believe this still sounds slightly ridiculous to me, I’m a survivor. I’ve actually been through a lot of shit and even when I acknowledge that I don’t really tend to give myself credit for pushing beyond it. Granted, I had a lot of help. I’ve been surrounded by a lot of tolerant, patient people that saw through the problems and helped buy me time to work through them. One thing in particular that I’ve forgotten about is just how much my biological father fucked with me when I was younger. (Not my Dad, John, whom I regularly miss and dearly love.) Here are some examples:

* When I was 2, my father left me on the sidewalk outside our condo and went to work for the day. A neighbor found me at or on my way to the Bohler Road MARTA bus stop and brought me home. This, perhaps unsurprisingly, prompted a divorce.

* When I was 3, he told me that Mom wanted a girl, not a boy. I was rather passive aggressive when I got home.

* When I was 5, I said to Mom, “Terry is emotionally unavailable to me, isn’t he?”. Frankly, I still find this a little hard to believe, precocious though I was.

* When I was ~8, I remember riding back from the liquor store in his Toyota Celica with some wine and him opening it and having me pour some into a cup for him. Classy.

I was also molested by a step-brother (Terry was unaware of this), suffered through a bit of military school in Boonville, Missouri where I was one of the smallest and youngest children, and probably some other stuff. So it wasn’t easy. I spent the first 18 or 19 years of my life not feeling safe, not trusting myself to connect with society or the world. When you get burned so much, why bother? But I came around. I’m remarkably functional, even warm. I *like* people. That’s kind of a fucking miracle. And there is some strength in me to get to where I am today. It’s something worth respecting and I’d do well to remind myself of that once in a while.

Tonight, I’m taking the opportunity to remember that with my blend of genetics and chemistry I could’ve wound up a total sociopath or just a non-functional, drug dependent nutter. I want to get healthier still and become an even more fully realized version of myself that isn’t as prone to bouts of low energy that I coast through. So I’ll figure it out and fix it. And I’ll achieve.

Read 2 Notes -Make Notes

Back A Page