Improved Means for Achieving Deteriorated Ends
22nd August, 2012. 8:34 am. Blogging Overhaul
Folks, this WordPress blog will soon disappear. I’ve wanted to get rid of WordPress for a while and half-wrote some blog software back in Spring 2011. Some coworkers had a similar plight so we all agreed to finish our blog engines in a one week sprint. I’m done and will likely switch over in the coming days. My livejournal, for those of you still reading that, will no longer receive crossposts and go dark. The blog will be at http://blog.redlinernotes.com/ and there will be an RSS feed at http://blog.redlinernotes.com/rss.xml. There is no ATOM support at this time but…you know, patches welcome. For the curious, the new software is called Coleslaw after a nickname of my favorite poet Milosz. It’s basically a jekyll-alike in Common Lisp. Anyway, see you on the other side.
17th August, 2012. 11:34 pm. A Fun, Maybe Useful Exercise…
A random walk through 30 years of history…
1960s: Lisp, Simula, Algol, our hallowed forefathers
1971: Unix released in Assembly, the advent of time
1972: Unix ported to C or, the advent of Portable/Commodity OSes
1973: ML interpreter in lisp appears, the advent of Typed FP?
1974: Deutsche+Greenblatt, the advent of LispM Dreams, fast implementations
1975: Sussman and Steele, the advent of Scheme
1976: Copyright Act of 76, the advent of Proprietary Software
1977: Apple II, Atari 2600, the advent of ‘hobby computing’
1978: 8086, The first BBS, TCP split into TCP/IP, the advent of networking
1979: CADR LispM paper published, the advent of Single-User Workstations?
1980: Smalltalk-80, the advent of OOP? (76 wasn’t released outside PARC)
1981: Symbolics LM-2 sells, the advent of the LispM
1982: Commodore 64; Sun-1 workstation or, the advent of the Unix Workstation
1983: MSFT announces Windows; Stallman founds GNU, the advent of free software
1984: Apple Macintosh or, the advent of consumer graphical PCs
1985: Symbolics registers first domain, NES, Intel sells 386, the advent of home computing? (spreadsheets, office suites, etc)
1986: IETF is formed, IMAP is developed, efforts to add OO to Common Lisp, more standards
1987: GNU Compiler Collection, the advent of free toolchains
1988: Soundblaster, MS-DOS 4.0, a boring year
1989: General Public License, the arrival of free software
1990: Haskell 1.0, Caml Light, the arrival of Typed FP? not quite yet…
1991: Linux 0.01, the advent of open source
1992: Alpha+OpenGenera, Linux GPL’d, Windows 3.1 is released, the advent of the Dark Ages
1993: Pentium I released, the advent of the “One True”(ly bad) Architecture
1994: Linux 1.0, XFree86, Red Hat, the advent of the distro
1995: IE 1, AOL 3 hits 2 million users, the advent of the net
1996: Linux 2.0 w/SMP, Apache leads web servers, the arrival of open source
1997: Wifi+HTTP 1.1, CD-RWs+Office 97, IE 4+google.com, nullsoft founded, the advent of the modern age
1998: US v MSFT, Netscape open sources Mozilla, the 6-year browser war purgatory
1999: Google Incorporates, the advent of Search
22nd June, 2012. 12:49 pm. An Emulator Design Pattern
Going into this project, I knew almost nothing about emulation. I still know very little. But I was tired of seeing emulators written in C and Java for performance or portability that wound up looking like a big switch statement sooner or later. My 6502 emulator is ANSL CL and should run anywhere that sbcl, clisp, ccl, or any other fine Common Lisp implementation will run. Granted, if a totally new processor architecture comes out, it’s probably easier to cross-compile/port a 6502 in C than the CL compiler hosting cl-6502 but I digress. I wanted to write a fast, high-level emulator. The closest thing I found to an emulator design that I liked was py65 but even that was a bit less abstract than I would’ve hoped.
I’m still searching for ways to improve the core abstractions that I have (and no doubt there are many hardcore low-level hackers that would be disgusted by my work) but I’m enjoying the process and here are some preliminary thoughts…
Addressing modes, addressing modes, addressing modes. This is the biggest difference between assembly language and the languages I use day to day. Hence, it’s a crucial abstraction to get correct. For the first time, Common Lisp’s notion of “generalized places
” has been a real boon. There were two nuances to addressing modes that I found I really needed to account for.
- Some modes access CPU registers, others RAM. (This mostly just effects the way I need to use setf.)
- Most opcodes use the byte /at/ an address rather than the address, but sometimes an opcode *does* need the address.
To solve these issues and abstract some code patterns, I wrote a macro called defaddress
. It defines a method on the CPU that returns the address computed by the mode /and/ generates a setf function that sets the register in the CPU struct or the byte in memory based on a cpu-reg keyarg to the macro. Finally, to solve the issue that we most often want the byte, opcodes are defined as either being :raw or not (the default). If they’re not, instead of passing the mode symbol to be funcalled, we pass a lambda that gets the byte at the address computed by the mode. So far, concerns seem nicely separated. Time will tell if I’ve struck on the right design here.
Opcodes in cl-6502 are really mnemonics, a set of opcodes that encode the same language primitive but for different addressing modes. As long as the foundation of addressing modes as funcallables is there to support the opcodes, you can write any opcode cleanly with a single body of code shared across all addressing modes. This has made me deeply happy as it seems to me to be _THE RIGHT THING_.
The implementation of defopcode
is a bit hairy, particularly the EVAL-WHEN
block to make sure metadata about the opcodes gets set in the *opcodes* array at load-time, but the supporting defins macro is fairly clean. The important thing is that opcode definitions wind up looking marvelous
. For example, here’s ASL and BCC:
(defopcode asl (:docs "Arithmetic Shift Left" :raw t)
((#x06 5 2 'zero-page)
(#x0a 2 1 'accumulator)
(#x0e 6 3 'absolute)
(#x16 6 2 'zero-page-x)
(#x1e 7 3 'absolute-x))
(update-flags (funcall mode cpu) '(:carry))
(let ((result (wrap-byte (ash (funcall mode cpu) 1))))
(funcall setf-form result)))
(defopcode bcc (:docs "Branch on Carry Clear" :track-pc nil)
((#x90 2 2 'relative))
(branch-if (lambda () (zerop (status-bit :carry cpu))) cpu))
ASL defines 5 methods here each with a different addressing mode but sharing the same body. They update flags in the status register as expected, increment the program counter properly, and put metadata in *opcodes* to aid with dispatch and disassembly. Not bad, eh?
Objects, Functions, Macros, Whatever!
So far, I’ve mostly just used objects (read: CLOS
) to define some conditions, two core methods on the CPU (step and execute), and the instructions themselves. The CPU itself is defined as a Struct rather than a Class. All the instructions are methods EQL-specialized on the opcode and the opcode alone which should make dispatch pretty speedy
. The methods reference the *cpu* global directly and since Common Lisp has /usable global variables
/ that look up the most recent binding in the current thread on reference, I should be able to run many instances safely in different threads on a single core. Just do something like…
(make-thread :foo (lambda ()
(let ((*ram* (make-array (expt 2 16) :element-type '(unsigned-byte 8))
&body))) ;; and we're off to the races!
I have a bunch of potential ideas for what’s next. I want to extend this work towards full NES emulation in the browser. There isn’t a formal ordering of priorities or milestones yet (cause this is kind of an art project), but coming up with a sane, RESTful API to sit on top of cl-6502 is probably the next step. Hopefully I’ll get some time to hack on that this weekend.
- cl-6502: An assembler! Unit tests + bug fixing.
- famiclon: An NES emulator backend built on romreader and cl-6502. Video, Sound. Input?
- Qeng-Ho: Hunchentoot+ST-JSON, REST API wrapping cl-6502+famiclon, No persistence! Multiple CPUs! Internal private API for now.
- Pham-Nuwen: Clojure+Clojurescript@Deepclouds.net. Persistence! Nice web interface. Graphics w/canvas! Etc…
- cl-z80: DO THAT SHIT! (how hard could it be? just another 8bit thing. see emu-docs.org)
17th June, 2012. 11:27 pm. On Interactive Retrocomputing
Lately, I’ve been working on an emulator for the MOS 6502 processor in Common Lisp that I’ve been boring enough to name cl-6502. The emulator is basically finished as is the disassembler. There are also pretty solid docs. An assembler should be added soon and hopefully a helper utility or two and unit tests. But why do this? Well, for a couple of reasons.
1) I never did enough Systems Programming. I never did any assembly in college and wrote an absolute paucity of C (granted, that’s my fault). I never learned enough about the inner working of Operating Systems or how to exploit memory hierarchies. I want to know more about how the machine works at a low level. Writing an emulator in a high-level language isn’t a great way to do that but I wanted to anyway. Writing some assembly programs to run on this emulator might help though and I hope to do some of that later.
3) ICU64/Frodo Redpill. I’m not wild about static types and I tend to write tests after the fact because I view programs as clay until they’re ready to be fired in the kiln. Roly Perera’s work on self-explaning computation interests me a lot as does Chris Granger’s work on Light Table. Rapid feedback loops are important. Maintaining flow is important. As far as I’m concerned, all emulators should strive towards the sort of “peek/poke the machine” experience that ICW/Frodo Redpill offers. Games are a very easy way to get people to engage with computers. Everybody likes games. And lots of folks at some point in wondering about programming, ponder how much is involved in changing something about a game they like. With a system like ICU/Frodo Redpill they could literally /see/ the answer. Add an integrated editor and you’re in pretty interesting territory. Feel like changing something about the “hardware”? Feel like having breakpoints and step debuggers pop up on arbitrary memory accesses or instruction executions? You got it. But ICU64/Frodo Redpill has this all locked down on desktops. Why not do as much as possible with HTML5, <canvas>, and Clojurescript? I’m not going to be the guy to come up with the next Mother of all Demos … but I hope to make something cool. And if I’m really lucky, I’ll have something interesting to play with online by Strange Loop on September 23rd. I’ve already got a 6502 emulator. What’s next?
16th June, 2012. 7:30 pm. A tentative Strange Loop 2012 Schedule
Strange Loop has posted their schedule for 2012 and my company has been kind enough to send me. Without further ado, here’s my current thought on which talks I’ll attend. I just can’t wait for September.
;; Sunday, September 23 (Emerging Languages Preconf)
– 7:30 flight? ZOMG WHAT WAS I THINKING?!?
10:30 Ostap Cherkashin – Bandicoot: code reuse for the relational model
11:30 Hakan Raberg – Clever, Classless and Free?
12:40 Michael Fogus – The Reemergence of Datalog
13:20 Brian McKenna – Roy
14:40 David Herman – Rust
15:50 James Noble – Grace: an open source educational OO language
16:30 Jose Valim – Elixir: Modern Programming for the Erlang VM
17:10 David Pollak – Visi: Cultured & Distributed
– STRAAAAANNNGE LOOOOP
;; Monday, September 24
09:00 Michael Stonebraker – In-Memory Databases
10:00 Dustin Getz – Monad Examples for normal people, in Python and Clojure
11:00 Pieter Hintjens – Software Architecture using ZeroMQ
– (or Functional Design Patterns – Stuart Sierra)
12:20 Neil Milstead – Augmented Reality and CV
13:00 Craig Kersteins – Postgres Demystified
14:00 Neha Narula – Executing Queries on a Sharded Database
– (or Clojurescript by David Nolen)
15:30 Scott Vokes – Data Structures: The Code That Isn’t There
– (or Lessons from Erlang by Garrett Smith, Types vs Testing by Paul Snively and Amanda Laucher) GAAH
16:30 Rich Hickey – The Database as a Value
17:30 Lars Bak – Pushing the Limits of Web Browsers
20:00 Matthew Taylor – Humanity 2.0
;; Tuesday, September 25
09:00 Jeff Hawkins – Computing Like the Brain
10:00 Chris Granger – Behind the Mirror
11:00 Nathan Marz – Runaway complexity in Big Data…and a plan to stop it
12:20 Carlton Mills – Computer Architecture of the 1960s
13:00 Oleg Kiselyov – Guess lazily! Making a program guess and guess well
14:00 Cliff Moon – The Audubon Society for Partial Failures
15:30 Ola Bini – Expressing Abstraction, Abstracting Expression
– (or Building visual, data-driven UIs with ClojureScript)
16:30 Bret Victor – ?
– And then I’m out because of my 7:30 flight. Sorry Brendan Eich!
6th June, 2012. 1:23 pm. 10 Great Hacking Albums
My posts on this blog have leaned away from technical content for the last year, tending towards the introspective and music or poetry. I’m hoping to start shifting back in the other direction for a little bit and plan to write about my latest personal hacking project soon. In the interim, here are 10 favorite albums to write code to off the top of my head. I will note at the outset that I prefer ambient and instrumental music for hacking. Ambient stuff in particular seems to naturally encourage a state of “flow” for me. Also, here’s a link to some C2 wiki discussion on flow as it relates to programming. Also, as long as we’re throwing out great hacking music I might as well shill the mixtapes I’ve been working on the last two months. All three are pretty solid. And there’s a one hour extended mix that mashes together I/Omega and Lost Without a Traceback that isn’t on soundcloud. Ping me if you’re interested. There are some track changeups and much improved transitions in the I./Omega half.
Tim Hecker – Harmony in Ultraviolet
Tim Hecker – An Imaginary Country
Tim Hecker – Ravedeath, 1972
Fennesz – Venice
Fennesz – Black Sea
Fuck Buttons – Tarot Sport
Rustie – Glass Swords
Araabmuzik – Electronic Dream
Four Tet – There Is Love In You
Amon Tobin – Supermodified
(Honorable Mentions: Tycho – Dive, Washed Out – Life of Leisure)
If you were to proceed through the above albums in sequence, you’d start with some fantastic downtempo ambient/noise stuff with Hecker and Fennesz, shift into uptempo “noisetronica” with the Fuck Buttons, transition into the over the top and in your face Rustie, ride those dancey vocals and synths into Araabmuzik, and then start winding down with the more midtempo vocals of There Is Love In You and the outstanding groove of Supermodified.
18th May, 2012. 8:00 am. 3 Years In
It’s been 3 years since Dad died, 2 years since I’ve been single. It’s hard to believe, really. And wouldn’t you know, I had a dream about Dad for the first time in a while. He had cancer but was still alive. We’d all moved back to Cross Creek where I’d spent the first decade of my life or so. I had my job at CMG, I think, and I was talking to Dad who confided in me that he and Mom were having serious financial trouble. I remember being shocked that something hadn’t been mentioned to me sooner and feeling guilty for somehow missing that I could have helped for so long. I’d just been enjoying life as I am these days, a good job, good friends, etc. I was talking to Mom or Dad about how I could support them when I woke up. In the dream, Dad still had all his hair. I don’t think I’ve ever had a dream about him post-hair and post-chemo. I’m thankful for that. He wasn’t the same person once his hair was gone.
I’d been a bit depressed the first half of this week and I couldn’t figure out why. It wasn’t until talking to mom Tuesday night that she reminded me we were coming up on the 3-year anniversary. I’d thought about it when someone asked about dad a few weeks back but it slipped my mind. I’m thankful for my job and coworkers, I love them dearly and even on my grumpiest, most “Office Space-y” days I’m happy to be in the office. I’m thankful for my friends who keep Atlanta fun and keep life interesting even when we all just want to have a drink and be together at the end of a long day. I’m thankful for my mother, who’s weathered plenty and still figuring out how to have fun. Mostly, of course, I’m thankful for dad. I still don’t know how he did it all. But thanks to him, I can too.
Read 1 Note -Make Notes
15th April, 2012. 4:21 pm. April Adventures
I’ve been working on some of the depression issues mentioned a few posts back, making progress. Been making a concerted effort to have fun. Lots of weekend trips to Athens, socializing, concerts, that sort of thing. Starting to create things again too! I’ve made a mixtape for the first time….like a proper mixtape. Did it in Audacity. Didn’t try to do tempo/beat-shifting or anything fancy, just 29 minutes of cutting and fading. I still think it turned out quite well for a first effort. It leans strongly in the electronic/dance direction so if that’s a big turn off for you steer clear. Anyway, onwards and upwards. And to steal from Milosz and Neruda: “Yes, this is my gift to you. Above ashes on a bitter, bitter earth. To you, to the one who unknowingly has awaited me, I belong and acknowledge and sing.”
Here’s the mix and the tracklist. (EDIT: Here’s a vastly improved version of the mix, the associated tracklist, and where you can listen online.)
Read 1 Note -Make Notes
17th March, 2012. 3:44 pm. A Quiet Saturday
I need another dose of Milosz. Perhaps you do too. Here we go…
In my dreams my guardian angel takes the form of a woman,
Not always the same one. He knows that I, a fleshly creature,
Need a lover’s touch. We don’t make love,
But there is a closeness between us, and understanding.
I never believed in the presence of angels, but my dreams have changed,
And when, recently, I found an underground grotto filled with treasure,
And we were moving the sacks together, I asked him
For one more moment of the dream, which gave me peace.
8th March, 2012. 11:39 pm. Cool Waters Ahead
Back A Page
Disclaimer: This is going to be a much more personal entry than normal. Just to get down some recent thoughts and revelations that I’ve had. I’ve read people talking about blogs as subject-focused, about building them almost like products. But this thing is a record, a mosaic, and a journal. So here we go.
On and off depression is the one thing I still really feel like is holding me back and don’t like about myself. Part of the reason I stayed single (even became single in the first place) was to address this issue. I’ve resisted taking medication since I went cold turkey in February of 2005. For the seven years prior to that, I had been on a daily cocktail of Tegretol (a mood stabilizer), Risperdal (an anti-psychotic), Effexor XR (an anti-depressant) and Adderall (ADHD med/legalized extended release cocaine). It was a non-trivial cocktail. I may take up a light antidepressant until I can learn to keep myself in the upswing without it. But this post is less about that and more about celebrating continuing to become mentally and emotionally healthier and making it as far as I have.
Let’s start with that last part. The meds I used to be on were *not* an accident and though I didn’t love the experience they had their use. I demonstrated many of the precursors for Bipolar Disorder and the primary reason I was on the meds was to curb the likelihood of my brain ingraining the sort of patterns that can only be fought by regular Lithium intake later. I have a close blood relative who has suffered from Schizophrenia and another who received 33 shock treatments over the course of a bad episode. I think we can safely say that things have panned out so far. I’ve been medication free for 7 years and all indications are that the meds steered me away from forming some pretty deep canyons for the chemicals in my brain to run through.
In addition to that, and please believe this still sounds slightly ridiculous to me, I’m a survivor. I’ve actually been through a lot of shit and even when I acknowledge that I don’t really tend to give myself credit for pushing beyond it. Granted, I had a lot of help. I’ve been surrounded by a lot of tolerant, patient people that saw through the problems and helped buy me time to work through them. One thing in particular that I’ve forgotten about is just how much my biological father fucked with me when I was younger. (Not my Dad, John, whom I regularly miss and dearly love.) Here are some examples:
* When I was 2, my father left me on the sidewalk outside our condo and went to work for the day. A neighbor found me at or on my way to the Bohler Road MARTA bus stop and brought me home. This, perhaps unsurprisingly, prompted a divorce.
* When I was 3, he told me that Mom wanted a girl, not a boy. I was rather passive aggressive when I got home.
* When I was 5, I said to Mom, “Terry is emotionally unavailable to me, isn’t he?”. Frankly, I still find this a little hard to believe, precocious though I was.
* When I was ~8, I remember riding back from the liquor store in his Toyota Celica with some wine and him opening it and having me pour some into a cup for him. Classy.
I was also molested by a step-brother (Terry was unaware of this), suffered through a bit of military school in Boonville, Missouri where I was one of the smallest and youngest children, and probably some other stuff. So it wasn’t easy. I spent the first 18 or 19 years of my life not feeling safe, not trusting myself to connect with society or the world. When you get burned so much, why bother? But I came around. I’m remarkably functional, even warm. I *like* people. That’s kind of a fucking miracle. And there is some strength in me to get to where I am today. It’s something worth respecting and I’d do well to remind myself of that once in a while.
Tonight, I’m taking the opportunity to remember that with my blend of genetics and chemistry I could’ve wound up a total sociopath or just a non-functional, drug dependent nutter. I want to get healthier still and become an even more fully realized version of myself that isn’t as prone to bouts of low energy that I coast through. So I’ll figure it out and fix it. And I’ll achieve.
Read 2 Notes -Make Notes