The Other Details

Whichclasses should I take?

2011-09-28T10:09:00.000-07:00

A project of mine got featured in the school newspaper!

Backstory: my roommate and I were looking for which classes to take for our technical electives. The University of Arizona site for reading course reviews is not very easy to use and makes it difficult to answer questions like "which class is the best in my department?". We scraped their data and made a better interface, it's currently at whichclasses.com so anyone can use it.

Normal Office Things

2011-06-10T23:37:00.001-07:00

San Francisco

2011-06-01T21:06:00.004-07:00

So, I got hired by Google for a summer internship. I'm working with the site reliability engineers on Gmail's spam, abuse, and delivery team.

Me in my Noogler hat.

Me in front of building 44's Android statue

The mountain view bowling alley

The bikes for getting between buildings

My desk (ergonomic keyboard is great, massive monitor is beyond great)

The San Francisco office slide

My cousin is working relatively nearby and I got a chance to hang out with him.

The Simplest Timer I Could Think Up

2011-03-27T21:00:00.002-07:00

Most of the projects I post up here are pretty software heavy, but I'm proud of this one mostly because I didn't write any code. On Friday, I walked into the shop for the race team I'm a part of and was informed that the timing system I was supposed to make was to be operational on Saturday morning, not Saturday night as I had expected. This made any plans of making complicated things with an embedded microcontroller completely unfeasible.

I did the only reasonable thing and immediately left the shop and went to Radio Shack (I was pretty desperate) and then to Walmart.

My goals for this were to make it as quickly as possible and keep it cheap and robust above all else, so it uses extension cords to communicate across the 150' run, a modified stopwatch to keep time, and the switches are mild steel boxes with aluminum levers.

This is my kitchen table at 3:00:

After some screwdriver time:

Added some leads to the switches on the inside:

Designing the switch pads for the car:

Welding up the first switch box:

Second switch box:

Adding some ramps:

Ramp reinforcement:

Two boxes done:

First switch done:

Both switches done:

Quick demo Friday night:

In action on Saturday:

The Multitouch Project

2011-02-21T22:57:00.008-07:00

I recently completed building a multitouch surface using CCV to track touches and provide a TUIO server and PyMT to provide a higher level GUI toolkit and Python bindings. It works using frustrated total internal reflection and I'm pretty happy with it.

The only thing written for it is a touch chasing game (video below) that I wrote with Kevin Luikens. It's probably going to stay like this for a while because the developers who wrote PyMT have moved on to project called Kivy which seems super cool, but is still in alpha. We don't really like PyMT and Kivy looks a lot better, so we'll probably start work up again as it gets more stable.

If there is something specific you would like a pic or video of, ask in the comments. On to the main event:

The Principles of Computation

2011-02-07T21:53:00.000-07:00

These ideas changed (and are changing) the way we think about everything and are why computing is beautiful to me. This is the best explanation I have seen of them.

A Brief Look At My AST Visualizer

2011-01-14T00:39:00.004-07:00

I have been working on visualization tool for Python's abstract syntax tree as a way to learn my way around the ast module. I needed to parse Python code to implement more context aware tracing in what I worked on over the summer with GSoC, and decided on using Python's actual internal representation of itself, instead of reinventing the wheel.

Making a visualizer which gets across the semantics of the program in a way understood by humans without knowledge of Python's internals has proven more difficult than I originally thought. It's pretty darn easy to get the project mostly working, but getting the little details right has led me to scrapping the two previous versions of what I had. Previously I had used recursive dictionaries and lists of various types to represent the AST because they are readily at hand in Python and writing transversal methods for them was pretty easy. They are too restrictive for graph rewriting though, which is essentially all this project is. I now have a version that uses a tree class I wrote for the project, which has proven more flexible but a little more quirky.

Non of this is important without pics, so I'll get on with it. I have been using the following program for my trivial examples:

 
    from math import sqrt
    a = 0
    b = 1
    c = 2

    def f(x, y):
        pass

    f( a + b, sqrt(c+1))

It uses the trickiest bits of Python semantics to get across, where the other representations had broken, in the minimal context to show them. The visualizer produces this right now:

This is far from perfect, but going in the right direction. There are still some awkward self-reference arrows (which are an artifact of my processing right now) and I should be able to cut out some of the nodes such as every instance of Name. I also need to differentiate between objects which are shared in memory but distinct in the program, such as the Add and None nodes. I'm looking forward to getting back on this and making it a usable teaching tool about how Python looks at itself. Making this so far has taught me how Python's grammar works very well and I am confident in building a block aware tracer when this is done.

Break Only Exists in Imperative Languages

2011-01-13T01:32:00.004-07:00

Perl: "Easy things are easy, hard things are possible"
Haskell: "Hard things are easy, the impossible just happened"

This break I had some pretty serious plans about finishing up by AST visualizer for Python. They were very quickly thrown out the window when I realized I didn't really want to think about school for a month and sleeping a lot was also a viable plan. However, I'm really bad at being completely non-productive. I ended up working through two fantastic guides for languages I've wanted to play with for a while. I worked through Learn You a Haskell For Great Good and Learning Perl, pretty thoroughly covering the spectrum of programming language practicality. I can highly recommend both resources, and plan on getting a review of Learning Perl up in the near future.

Haskell is beyond incredible. I love the feeling of grasping entirely different ways of looking at problems and I got it almost every chapter of Learn You a Haskell. I enjoy the functional aspects of Python (especially closures) but working fully in that paradigm takes a bit of getting used to.

The breadth of features is difficult to describe succinctly. Full power lambdas (anonymous functions) are really nice to have and language support for partial application makes working with higher level functions really easy and often makes lambdas unnecessary. Folds are in Python but I really didn't get the point of them as an abstraction of the basic data-structure transversing recursion pattern until seeing their widespread use in Haskell. Typeclasses (most similar to interfaces or abstract classes in OO languages) make some really awesome abstraction possible across a wide range of data types. Trees are foldable just like lists, minBound and maxBound can work on days of the week, all sorts of things can be mapped across like lists, and any user-defined data type that implement the right functions can work with the built in functionality just fine.

I feel about monads right now exactly the same way I felt about pointers in C when I was learning to program. I'm surprised they aren't as scary as everyone makes them out to be, I see they are incredibly powerful and useful across a huge domain of problems, and I don't really see everywhere they are useful or how to make them and work with them outside the confines of tutorials and canned problems. They are, however, an incredibly interesting approach to a pattern seen throughout programming.

On the whole Haskell is the most strongly typed language I have ever used, but the typing and compiler checking don't feel tedious or draconian, they help me both think about the problem I am solving and implement the solution. I haven't written any non-trivial programs in Haskell yet, but I very happy to have spent time playing with it. Python is no longer the most expressive language I have used.

Weird Things Happening

2011-01-03T10:47:00.001-07:00

I accidentally rebooted my computer yesterday and the battery charged overnight, when I woke up today both the acpi command and htop are really fast at all things battery. I'm going to talk about something else for a while, because it seems like I don't have a complete understanding of what influences /proc access speed.

Edit: As of the day after this post, everything is really slow again. I need to do some more research to figure out what is going on.

Into htop's Source

2011-01-03T03:12:00.001-07:00

I poked around in htop's source today and found the following incriminating lines in BatteryMeter.c:


   double percent = getProcBatData();
   if (percent == 0) {
      percent = getSysBatData();

      if (percent == 0) {
         snprintf(buffer, len, "n/a");
         return;
      }
   }

That looks like my theory is confirmed. getProcBatData(), the only one of those functions that should be getting called on my system, opens at least two directories and two files every time through the main loop. This blocks the switch statement that processes character input, thus making htop sluggish. I didn't think that file operations were that slow, especially in C, but maybe things are different in /proc and /sys due to their unique relationship with the kernel. I'm going to poke around a little bit more tomorrow to get a good handle on how everything is behaving before I send anything to the developers. My first instinct is to try to keep open the correct directory and only reopen the needed files. I don't know how this will affect things and should probably run gprof before concluding anything. An alternate solution that seems pretty clean to me is running a helper thread that reads the data from the special file systems so if it takes a while to read, the battery data just updates slowly, rather than blocking the user interface. I don't know if that would fly with the developers though, I don't see any other thread usage in the program.

Problems With htop: Fixed.

2011-01-02T02:09:00.000-07:00

I really like the command line activity manager htop because it is super light-weight and much more useful than top or ps. I started using it last year, but it suddenly got unbelievably sluggish to startup and respond to key commands about six months ago. A tool like htop needs to be responsive because I use it primarily when something has gone wrong and is leaking memory or I overloaded everything and need to lighten up the load, so replacing it with the GUI activity manager supplied with the system isn't a great idea. I tried to use my favorite troubleshooting tool to see what was slowing it down, to no avail.

Today, I grabbed another user's config file today on a whim and lo and behold; htop was snappy. I diff'd that config with mine and applied the changes one at a time until it sped up, finding the problem in an unlikely place. What ended up triggering it was the option to display battery state. I think the problem comes from checking /proc/acpi/battery every time through the main loop. I don't know exactly how those files are generated, but they take forever to read and htop runs in a single thread so the file operations block the user interface. I am going to see if I can confirm my theory with the source code tomorrow and possible drop a line on the developer's mailing list. For now, I just removed the battery from what htop reports.

I like having command-line access to my battery state (the icon in Gnome is pretty buggy), so I replaced it with a tool aptly named acpi, from the Ubuntu repositories. It also takes forever to read out state, but it only does it once and doesn't tie up an entire useful tool doing so. If you really want to, you could probably throw something together with the watch command to make it run continuously, but I really don't need to see how my battery is doing that much.

Blinky-Flashy Time

2010-12-03T01:42:00.002-07:00

One of my friends picked up a strand of these and brought it in to the IEEE lab/hackerspace for us to play with. This is what happened with five minutes of collaboration after I messed around with the capabilities for a while. I really like the colorful LEDs and the Arduino makes coding for them really fast and easy. It's the same feeling I get from programming in Python, but for microcontrollers. We have some more ambitious projects planned for those parts bins, I'll keep everyone updated.

I'm really feeling this right now.

2010-10-23T18:31:00.003-07:00

Summitted the Code To Google

2010-09-09T23:55:00.000-07:00

I just submitted my sample code to Google. I have a lot of work right now, but will have a longer post describing my feelings on the project a month after I finished coding and get some updates on my current work.

Parrot 0.1.0 Released!

2010-08-17T16:51:00.000-07:00

Yesterday I buttoned up my work and put my pencil down. I finished up its exception handling abilities and added some more examples. You can download the release archives here. I'm laying pretty low this week, as I prepare to go back to school next week, but I will be sure to get a post up reflecting on the project within the week. I have another big project (or two) right around the bend and will continue to post updates when they get going. This has been an incredible experience and I am really grateful that Greg and Google supported me in working on it.

I Have Name

2010-08-16T00:18:00.000-07:00

With the help of a lot of advice and brainstorming, I have decided to call the project Parrot. Tomorrow I will button up the project, tag a release in the hg repo, and produce a tarbell. This has been an incredible experience and I will get a more in depth reflection up soon.

Redoing The Examples

2010-08-12T23:29:00.000-07:00

I'm going to keep this short, I've been typing a lot today, but I wanted to get an update in. I reordered the buttons to make them more natural and am making some more comprehensive examples. I have one last issue I would like to fix; I want the syntax checker to be triggered when a trace is created. I should get all of this buttoned up by tomorrow or shortly after and be able to make a release of it after I go over the code and refactor it with a fine tooth comb one last time.

My Wrist Feels Better, In Theory

2010-08-11T00:05:00.000-07:00

My wrist surgery went really well today, so I think I will be able to put in some work tomorrow. I really don't have much left to do besides make some more comprehensive examples and refine the way the Create Trace command interacts with the user. Today, I really didn't do a lot because I spent most of work time in the day munching pain pills and taking a pretty serious nap. However, I did get something fixed in the difference highlighter for the variables fixed. Before it would only highlight the last variable which changed, instead of all the variables which changed because the general purpose highlighter function I wrote cleared all highlighting before highlighting the line (I defined it primarily to work on the source, at first). Now I added a parameter to the function so it will not clear the highlighting if that is specified false, which solves my problem. Now when a context changes (such as entering or leaving a block with its own local variables) all of the variables are highlighted and when multiple values change it is visible that they all changed instead of just the last one which changed.

Final Tweaks For A While

2010-08-08T19:26:00.000-07:00

Today I got a lot done and mostly finished off my list of things which bother me about the software. I cleaned up unused code in the trace module, which was just sitting around and I hadn't noticed. This lead me to finding an old TODO comment I had left for myself, reminding myself to fix an issue which lead to the trace window appearing to hang briefly every function return. So I fixed that. I also noticed the globals and locals display windows weren't working properly, which was related to some code shuffling which I hadn't noticed. I fixed that too. I also got the font and size configuration to work across the whole system.

Tomorrow I will be traveling back to Arizona for most of the day, so I don't expect to get a lot of code done, but I will be working on some examples as I need a lot more. I'm going to have surgery on my wrist on the 10th, so I'll be out of commission for a while. I will spend that time thinking of a name so I can have a snappy title for the first release I put up on Google code.

Almost Done

2010-08-07T19:15:00.000-07:00

After today, I only have one major and one minor issue left until I can package everything up and release it as completed. I need to make the menus more applicable for the trace display window and I need to make my code read the configuration files instead of having hard-coded values. Nothing I don't know how to do.

Today I went through my code and documented a lot of it (though I would love some tips on making good docstrings) and cleaned up the cruft. I found a couple of places where I didn't use the functions I made for accessing the Text boxes in the trace windows. This made the code more complicated and fragile than it needed to be. I also found some places where I almost duplicated code, so I combined those areas with some minor logic changes. In the process I found a minor bug which was hurting the user experience and fixed it, streamlining a couple of other sticking points along the way. (Take home lesson: code duplication is evil.)

I also added a couple of minor features to address some things I found user testing. The titles of the windows are much more descriptive for someone who isn't already a programmer, stderr now prints in red to my output window, and variables are highlighted when they are changed in the local and global variables windows. All are things which I think will make for a much better experience.

Tomorrow should be very productive, hopefully I will get everything done soon enough to take a good look at the module which actually does the tracing. I inherited some of the code from Geoff Salmon, who worked on a related project before me, and I fiddled with it quite a bit, making it not as polished as I would like it to be. If that gets all done, I will make more examples to show the new users what can be done with this project.

Fixing Bugs, Finding Bugs

2010-08-06T20:44:00.000-07:00

Today was really productive. I got the class integration completely finished and everything works much better than when I started. I did find, however, that the incomplete integration I was previously working with masked some issues such as the fact that every time a new PyShellEditorWindow (the is the standard editor window which the IDE is based on) is opened my changes to the menus are reapplied, this gets awkward after a few windows are opened. I spent the rest of my work tracking down and squashing a couple of UI bugs which I have been unhappy with for a while, but didn't want to devote time to until I had a more mature product.

I've been repeatedly running into a problem working on this; I have a difficult time deciding in which methods to put the logic governing the alternate work flow I've created. I don't know if this is just an issue with my immaturity as a software architect or if IDLE is just very strongly structured towards one idea of how it should be used and modified. I can see in many places where I think the design is awkward or obscure the designers were fighting with the limits imposed on them by Tkinter so it might be caused by something else all together.

In other news, I was shown Spyder by Robert Erdmann, a professor of materials science and engineering at the University of Arizona, and I really like it. I haven't had a chance to look at the code yet, but it seems like an excellent tool for beginners (who may be turned off by the command-line orientation of Python) and one of the better Python IDEs I have seen in a long time. I will be continuing work on this project in the fall as an independent study advised by Dr. Erdmann so I can extend the capabilities of the tracer and its markup and study how the beginning programmers in MSE 350 respond to it as a learning tool.

OO Gets It Done With Less

2010-08-05T21:09:00.000-07:00

Right now I am midway through transitioning my work into the natural class hierarchy of the program. I think that basing my new window class off of the existing interface useful, despite the fact that I don't need all of the features it provides. The actual GUI is very different, in fact using a completely different geometry manager, but the input and output still need to be managed and configured by the rest of the system. By subclassing the base class of all of the major windows, I got relatively easy access to the file I/O subsystem and the existing configuration system and no longer have any extraneous exceptions coming from class mismatch problems. I got around the GUI problems by unpacking the old GUI and reassembling it using the grid manager with the other components (which the base class doesn't know exists) after calling the __init__ method of the base class. This isn't the completely optimal solution but it is much, much better than what I was previously doing. I still have some work to do to get everything back to the way it was, but the take home message is I removed almost 600 lines of redundant code today by using a subclass instead of rolling my own class.

On The Home Stretch

2010-08-05T08:09:00.001-07:00

I forgot to post last night, so here's what's happening: I have four days of coding time left to polish up the project and get everything presentable. There are a few issues which are still bugging me which I will be concentrating on. Most importantly, I just duplicated a class and modified it for my own needs to make part of my system. That class needs to get integrated in with rest of the IDLE so there is less grating at the interface and less duplication of code, thus making it easier to accomplish the next goals. The menus need to be redone for the window which displays traces so they actually correspond with what is being done in that window. I need to make opening files happen in unified, coherent manner. Finally, I need more examples which demonstrate the capabilities of the system.

As my work goes on, I'll post what I have refactored, what I changed, and why I changed it that way.

New Screencast!

2010-08-03T19:01:00.003-07:00

Please click through to see it full size.

Refactor to Refactor

2010-08-02T21:22:00.001-07:00

Today I reworked my GUI to use Tkinter's grid geometry manager rather than using the packer. I ran into some issues when I was user testing and was working on fixing them when I realized how incredibly complicated my fix was to implement. Refactoring time it was. I spent the day playing with the grid manager trying to replicate my previous work. I don't have it all working yet but I made a mock-up to get everything working without dealing with my existing code. Here it is:

Also, why do I think not taking to the time to do something the quality way isn't just a waste of effort? Every time I let my code quality slip in the name of efficiency and "it's just a little one-off to figure things out", I end up reworking all of it so it is modifiable and understandable. Best practices got that way for a reason, so no more shoddy work for me, even if I'm the only one who will see it.

Interesting Feature in Python

2010-08-01T20:14:00.001-07:00

I've been reading the documentation of Python's internals, trying to get a better feel for how it works and compilers are implemented in general and I found these lines in the grammar: "while_stmt: 'while' test ':' suite ['else' ':' suite]" and "for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite]". These say that

for i in range(4):
    print i
else:
    print "this isn't a syntax error"

and this

x = 4
while x > 0:
    i -= 1
    print i
else:
    print "this isn't a syntax error"

are a perfectly legal bits of python code. They behave a little differently than I first expected. The else block executes in all cases except a break statement within the looping construct. I understand why I never heard of these when I was learning Python and I probably will never use these constructs, but now you know about them, so you can continue not using them as well.

SCREENCAST!

2010-07-27T19:40:00.002-07:00

Fixed The Big Bug

2010-07-27T12:21:00.001-07:00

It took four days of education and two lines of code, but it's fixed. Cool examples to come using my new tracing files with locals imports skills.

This Exists

2010-07-26T21:57:00.002-07:00

I really like Wikipedia

Very Good Day

2010-07-26T20:32:00.001-07:00

Today I got all most of the usability problems fixed, got some working examples committed, and a screencast recorded. I found a pretty good tool for screencasting, but it only outputs flash files. I'm downloading tools to fix them, but I will be asleep before they are done. The screencast will be online before I go to bed tomorrow. Furthermore, I have the program in a place where I am comfortable with people testing it, as I feel most problems they have will be the result of design problems, not execution problems. I have also finally contacted the IDLE developers about how they see my project fitting into IDLE.

At this point, the only major outstanding bug (that I know of) is the issue with tracing into included source files, but I will be attacking it full bore again tomorrow.

I found a gem.

2010-07-24T20:40:00.001-07:00

This is really cool and everyone should install it immediately.

(Edit: If you alias that to Python, make sure to unalias it before you start playing with sys.settrace or it will trace the mainloop which is way too much information and doesn't end well.)

What's Been Going On

2010-07-23T22:02:00.000-07:00

The last post on here was very optimistic, however I have gotten very derailed. Since my last post, I have been wrestling with a very frustrating bug. The tracer I have been using chokes when it imports certain types of modules, such as the ones Dive Into Python uses for it's non-trivial examples. It seems to be fine with small modules and all of builtins I have tested, but it really freaks out and starts acting very strangely when a large local module is imported. My hypothesis right now is that it is tracing into imported modules (though I don't know what is causing this) and this is chewing up a lot of memory. The first cause is simply the wrong behavior, the second is a serious program issue. Hopefully fixing the first will cure the second. I have put off a couple of things which I was planning on doing, because this seems to be a much higher priority problem. I'm going to hit it really hard this weekend (along with minor usability issues which I have been documenting) and have been collecting symptoms to try to get what is causing this worked out. More details as this unfolds.

Moving Forward

2010-07-22T00:00:00.001-07:00

Today was pretty slow, but a lot of things are ready to move forward. I attempted to make a screencast this morning, but ran out of ram, which kept my program from executing at all, making the screencast really pointless. I will try to get a machine with more resources tomorrow so I can finally get a screencast up and show the world what I have been up too. I started working on examples and noticed some usability issues, starting my list of things to tweak. I worked on translating the Dive Into Python code which is going really fast, I should have the first round committed by tomorrow. Also, I got a great response from some of Greg's grad students today to help me with testing, which I really appreciate. My planned activities got cut a little short today, because I stopped working around 3 to go to DC and meet up with my Dad, who I haven't seen in a couple of months. I enjoyed myself a lot; it was really great catching up with him in person. Tomorrow, I get cracking again.

Major Prototype Work is Done

2010-07-20T21:05:00.000-07:00

I just finished all of the major features of my prototype and should have a screencast up tomorrow morning. I've spent the last couple of days reading a lot of code and writing a bit of code, but deleting most of it as I tried to figure out a decent mechanism for getting around the limitations imposed by the subclass-superclass relationships in IDLE. I've learned some cool tricks in the process, about what sort of things can be done with Python classes, which is a decent upside. There are a few little tweaks that still need to be done, but as soon as I get a screencast up, I am going to contact the IDLE-dev mailing list about how this should be incorporated into IDLE and what it's role is. I really, really need some UI test participants at this point. Hopefully I can find some soon, or I will have to resort to wandering around the Hopkins CS department ambushing anyone who might know Python. Please, dear readers, don't let this happen. If you are anywhere near Baltimore or know someone who is, who would be interesting in playing with my software for half an hour, send them my way.

Happiness is a Warm Internet

2010-07-20T10:51:00.001-07:00

I had an awesome weekend, but a frustrating couple of days since then. However, the frustrating stuff is starting to clear up. (a little)

First frustration, I haven't had internet access at home for the last couple weeks and because I am subletting, I had to depend on other people to get it working again. It was supposed to be up by yesterday at 5:00 p.m., but as I right this it is still not up. A serviceperson was just out from Verizon and got the connection working, however I need to successfully navigate their installer before I can actually use it. The Verizon high speed internet installer is a thinly wrapped javascript-in-a-webpage application which will only run on Windows or OS X (it gave me a .exe, on Ubuntu) despite executing entirely in a web browser. It relies on the name of the executable for verification (exploits here anyone?) and is a shining example of the importance of UI testing. Namely, large companies should really invest in it before they foist a terrible product on their customers and put a huge barrier between their users and their actual product (an internet connection). Furthermore, they have the gall to ask if I want to install the 'In-Home Agent' to help with getting computers connected, setting up my network, and getting the motivation to slam my head through a wall at the name 'Verizon'. I have faith that the helper agent is just as worthlessly unusable as their installer. When I decline to install the extra-helpful additional software, they hit me with poorly punctuated popup boxes/sales pitches, which is just lovely. Finally, the program crashes, and prints out what appears to be a traceback because a JS function got called with too few arguments and a cryptic error code. Then when I close the window, it opens another popup box asking why I quit. (I choose technical issues.) Right now, my girlfriend is listening to loud fuzzy fusion jazz while the Verizon help line tries to get around their own software bugs. Hopefully this will be done by tonight so I will not have to go for a hike every time I want to connect to the internet and can stop this wild goose chase. (We just got everything working and the Verizon Download Manager is continually crashing my girlfriend's web browser, sweet.)

The other frustrations are actually related to work--that thing I'm supposed to be doing when I'm not on the phone with Verizon--but it will have to wait for another post, because that rant was enough for one post.

I'm Silly

2010-07-15T18:10:00.000-07:00

So... I forgot to push my work to the google code repo before I left. So now I can't do any work on the computer I brought. Oops.

Good stuff come Monday, I promise.

Fully Functioning Prototype is Working

2010-07-15T12:27:00.001-07:00

I just finished up the major work on my revised prototype and am very happy with it. I will spend the rest of today working on polishing it up, tweaking it, and putting in niceties such as error handling. Everything needs to be throughly cleaned as it has gotten pretty messy during my build effort, but when it is done I will get everything committed so anyone can use it. I really need some people for user testing now. I'm going to NYC tomorrow and will spend the weekend there, so I won't be working on this project. However, after I get back, I will get a narrated screencast up.

Science!

2010-07-13T13:29:00.001-07:00

I spent the day with my friends, in their p-chem research lab. I was a little distracted (science is too awesome to completely tune it out), but I got a lot of stuff polished from yesterday and removed some of the hacks I used to get everything going. I also filled out my mid-summer evaluation for Google today. In other news, I found a copy of Mathematical Structures for Computer Science for free in a hallway, which makes me really happy.

Storms!

2010-07-12T14:47:00.000-07:00

I was not as productive as I would have liked today, as we had a really bad storm and the power went out. I'm working on making a cohesive IDE with my various little prototype chunks, which should be done by the end of tomorrow. I'm also working on converting a few Dive Into Python examples into traces, for use with UI testing, more screencasts to come.

I Can Parse Annotations

2010-07-09T16:31:00.001-07:00

I wrote a function today which will strip the annotations out of a file and make a dictionary mapping the line numbers of the new file to their annotations. It is pretty fragile right now, and a little more complicated than I would like, but it works. I only have a little more work to integrate it with the parser and have it generate annotated traces for me. Then, all that is left is to make a fully functional recording and playback environment. I'm very happy with my progress right now.

Grinding a Little Bit Closer

2010-07-08T14:16:00.002-07:00

I really like JSON, however I really don't like writing it out by hand. The program I am working on uses JSON execution traces, so any new functionality I wanted to add had to be hacked into an existing trace by hand. To fix that problem, today I extended the code tracer which Geoff made and is available here so it would include global and local variables as well as the execution flow. It's not quite done and needs some general polishing, but it was the last major piece keeping my prototypes from being fully operational, so this is quite a relief. It took me a while to figure it out, it uses some really useful features of Python which are really just thinly disguised black magic, but tomorrow it should be feasible to add annotation parsing to it, which will allow a work flow like I described in this post. With just a little bit more work, the system that undergoes user testing will have very similar features to the final product!

Screencast is here

2010-07-07T15:25:00.003-07:00

This is a recording of what my program can do so far:

I will get captions on the video as soon as I figure that out.

UI Changes

2010-07-07T14:16:00.002-07:00

I was thinking about usability and how this projects could be improved the other day and noticed that the approach I have been taking is both too simple and two complicated. The two pane model does not associate specific annotations with the lines they are explaining well, either from the perspective of someone writing annotations or someone using them in the context of a tutorial.

Instead, annotated files could be produced using a standard IDLE editor window, generating a trace through the usual means (menu, keyboard shortcut) or any text editor with the result run through a filter style cli tool. This would open up how file generation is accomplished and make the associations between annotations and program text much more apparent and interacting with them much simpler.

This also means that the playback window which a student would use can be dedicated to that task. A dedicated window opens up several possibilities. It is now possible to implement a button interface, which would be intrusive in a non-dedicated workspace. This makes it easier for new users to quickly start working, without spending time learning the program. Finally, a dedicated playback window would allow a specialized window layout, as it is only used for playing back traces, not creating them.

Take a look for yourself:
Old Way:

New Way:

Potential Use Cases

2010-07-07T08:16:00.008-07:00

These are the use cases I would like my test participants to use when doing my UI testing. I would love to hear any feedback.

Case 1: You are making online tutorials to teach software concepts, and would like a way for you students to get the same experience as they would sitting down with a teacher. Open the file "example1.py" and annotate it with explanatory comments for your students, generate a trace file from your annotations.

Explanation: This is the usage pattern which accompanies transitioning a group of examples or tutorials to this system. This is the most important early case to nail. By making it very easy to use this system, it will be easier to gain acceptance among teachers. If teachers won't use this, it doesn't really matter how useful it might be.

Case 2: You are a student who is learning to program from online tutorials and have downloaded the file "example2.jtr" as a supplement to one of the tutorials. Using this program, open the file and as it executes, describe what the program is doing.

Explanation: This is the most common usage among students using a tutorial. Each group of lessons created using this tool, is made by one or possibly a handful of people. However, there is no cap on the number of people who could use those lessons, so the vast majority of users will only interact with this aspect of the program. It must have no major obstacles, or it will drive people away from lessons.

Case 3: You have a previously existing program file which has been annotated for use in teaching and you wish to change it to correct an error. Edit the program and its annotations to correct your error. Then, generate a new trace file which you can distribute to your students. Run the trace to verify its correctness.

Explanation: This is the edit/eval loop of this process and it must be easy or example files will stagnate, not being kept up to date or accurate. If it is not easy to progressively tweak existing files, they will be abandoned or not created in the first place.

Case 4: You are a teacher who has received a trace file from one of your colleagues. When you run it, you notice a spelling error in an annotation. Open the trace in your favorite text editor and correct the error in the trace.

Explanation: This is the least common usage, and could be omitted if the testing needed to be cut. However, it is important to verify that they intermediate data file is understandable and easily parsible by humans using common tools. So that small tweaks could be possible and the trace file could be processed by anyone. The JSON usage will help this.

Today, in a docstring

2010-07-05T12:38:00.007-07:00

>>> from types import CodeType 
>>> print CodeType.__doc__
code(argcount, nlocals, stacksize, flags,
         codestring, constants, names,
         varnames, filename, name,
         firstlineno, lnotab[, freevars[, cellvars]])

Create a code object.  Not for the faint of heart.

More Books, More Learning

2010-07-02T19:42:00.000-07:00

Today I read Design Patterns a lot, and thought about object-oriented design a lot. I also got the trace display tool mostly working with IDLE, which means everything needed for a student to use my prototype to learn things is done. I am going to start usability testing as soon as I can get some participants. If you are living near Baltimore, and would like to try out some new software, I would love to hear from you.

So, Testing....

2010-07-01T21:19:00.001-07:00

Today, I didn't get much code written because my books came, which was awesome, because books are awesome. So without further ado, another book review:

I just finished Rocket Surgery Made Easy, and if you are remotely interested in usability, get it. Stop reading and get it.

If you are slightly less impulsive and want to know some more, I can oblige. Before I got this, I knew a little about usability, but didn't really have much of a clue about how to go about testing. He delivered some very pragmatic advice, gleaned from quite a career in usability and user interface design. I specifically liked his advice to start earlier than you think is reasonable. This makes sense, all problems are easier to correct when no one has implemented them yet. His advice on fixing problems through tweaking and making the smallest possible changes is also helpful. A total redesign takes a lot of work and puts you back on square one. A tweak is likely not to mess things up too much, and doesn't need a lot of resources to get done. (Also making it more likely.) Overall, his message is very pragmatic and doesn't present anything that would be impossible or particularly difficult for anyone to implement, he just offers some gentle guidance to avoid the pitfalls.

I didn't have any real issues with content, however I did find the layout, font, and color choices to distracting at times, they can be very busy and make it hard to focus on the text. This really isn't that big of an issue as the whole book feels like a website, really. It is very lighthearted and one of the funniest design books I have ever read. I will definitely keep it close by as I begin user testing.

I will review Design Patterns as soon as I am done with it. (It's a little more...academic, so that might a bit more time.)

Engineering Students: Machines Which Turn Money Into Fun Stuff

2010-06-30T20:12:00.002-07:00

I don't really have a lot to report on the software front. I fixed some little bugs (and one bigger one), but today was mostly a learning day for me. I bought a copy of Design Patterns and Rocket Surgery Made Easy from Amazon (the first because it was on sale and I wanted it, the second on Greg's advice). So, as soon as those come I should have some great reading to do. I am also very excited because I bought a bunch of components to play around with for embedded projects, which are really, really cool, but a lot more expensive than Python code or the GCC. I will get a screencast of what I am working on for GSOC up asap and hope to have an integrated system out of my prototype by the time Rocket Surgery comes in the mail so I can start UI testing and revision. Also, I need to figure out how to write "Hello, World" in Morse code, or a lot more LEDs.

Another Little Testing Program

2010-06-29T21:17:00.001-07:00

I made this today:

It's a python version of this, which is made by Geoff Salmon--a former grad student of Greg Wilson--as a prototype/addition for what my GSOC project is trying to solve. Both of these programs take a JSON program execution traces and allow you to step through the code and watch it run. When coupled with my annotation facilities this will allow a full explanation of code from within IDLE. Right now the step back function is a little sporadic, but the code will be released to my repo as soon as that is fixed.

Truth Delivered In Comic Form

2010-06-29T08:27:00.004-07:00

#634; Accomplishment measured in Decibels

Yes.

Onward Ho!

2010-06-28T20:45:00.000-07:00

I don't have a whole lot to show you today, but I got a lot accomplished. My modified IDLE can now record and fold annotations and most of the irritating bugs are worked out. I also found a better user interface model. It is easier to code and less cumbersome to use, but is a little bit more fragile. After that all got done, I worked through making another little test bed for the traces generated from this tool. That should be finished up by tomorrow and I will have some screenshots and something to play with by Wednesday, when I combine this with my larger testbed.

Success

2010-06-25T15:14:00.003-07:00

Yep.

The Simplest Things

2010-06-25T12:20:00.000-07:00

The reason I was having such strange and inconsistent issues yesterday is because Text widgets have a 'name' attribute and any widgets with the same 'name' are treated identically. That's silly.

Truths

2010-06-24T21:06:00.000-07:00

Things I know about adding more Text Widgets to IDLE:

If they are both in the same frame, only one will show up. However, any insertions directed at either will show up in that one.
If they are in different frames two will show up, but you get to pick which one responds to commands.
If you try to get them all two, the program crashes, citing problems about a tag which doesn't appear to be referenced in the command assignment.
The previous points all apply if you try for three.
Any attempts to track what is happening in the background will lead into internally networked code, dark tcl hackage, or both.
The process is frustrating.
I have a lot more work to do tomorrow.

A Tour of the Testbed

2010-06-23T15:15:00.004-07:00

Today I worked on getting my code from the testbed where I built it into the actual IDLE code, which is frustrating. So I'll show the world what I have so far, because it is a lot more interesting (click on the pictures to enlarge them).

This is how it looks when it loads a file with annotations, the long names are folded. (I set the folding length short for this demonstration, so it is easy to see)

This is the same text, unfolded.

This is how you go about editing the annotations, mostly because it was convenient to implement this way.

This pops up to perform the edits in.

And this is the result. I think it needs some way to visually line up annotations with source code better, but this will work for now.

Sweeping Up the Shop

2010-06-22T20:30:00.000-07:00

I didn't get a lot of notable stuff done today, mostly just squashed some bugs that were bothering me a lot and worked on integrating my test platform with the actual IDLE application and the tracer. More to come as that progresses.

A Prototype is Born

2010-06-21T19:33:00.001-07:00

I got it done! It will open up any python file and recognize formatted annotations. The annotations are then separated out and displayed next to the source code. Though the program is very brittle right now and has a few bugs to work out, it performs it's essential function. All it needs is to be joined with a trace generator and trace reader and it will be able to perform the essential playback duties required of it.

If you want to take a look at it you can download the repo and run fold_lines.py. To fold/unfold annotations, right click on the annotation and select fold/unfold. To edit an annotation right click on the source line it annotates and select Edit Annotations. I'm not very happy with the UI, but it gets the job done for now. Lots of polishing to go on tomorrow.

Everything works! (As long as you don't try too much)

2010-06-20T20:42:00.000-07:00

I got my folding and annotation mechanisms connected today and found this. I'm happy someone else has tackled the synchronization problem, which had me stuck most of the weekend. I will be thoroughly mining that recipe for inspiration, as its solution is different than mine, and works much better. It also gives me some clues for solving my other synchronization problem so if I bust my hiney tomorrow, I should be able to make this work completely very soon. Also, I'm very sorry to my loyal readership, but screencasts aren't working very well for me. I've tried record-my-desktop (doesn't update the entire window at once), instanbul (hangs), and xvid screen capture (crashes); if anyone has some tips for me, I would love to hear them!

Lots of parts, not really assembled yet.

2010-06-18T22:34:00.000-07:00

Today, I got source code parsing working! Right now I have fully working folding and annotation for static files. They still need to get integrated into the split text widget and made to all work together in a way they could really be used. Tomorrow I will post a screencast of what I have working. In other news, Lily dyed her hair purple today and it was really fun. Google code has the most up to date code if you want to check out my test platforms for each feature.

Stuff is WORKING!

2010-06-17T08:44:00.000-07:00

I got full featured folding working this morning and it just needs to be integrated into the main widget and that aspect will be done! I am looking forward to OrderedDict becoming a permanent part of collections, as it made my job a lot easier. I still have a little bit left on the main widget, but I'm going completely stir crazy and am exploring the city after lunch and then going to a poetry reading to try to regain a bit of my sanity. I'm feeling really good about the project right now, but I really need to get out and take a break from thinking about it 24/7.

Steps Forward, Steps Back

2010-06-16T19:33:00.000-07:00

So, today I thought I would start on this before I was completely delirious to cut down on the amount of editing I would need to do after I woke up and read what I wrote. I got my widget to behave like a Tkinter.Text and got a basic folding mechanism implemented. Also, I figured out how to implement linking the annotations with their code, but I don't have it coded yet. Still on the plate is a weird conflict between my widget and IDLE proper as it isn't showing up yet and I don't really know why. I'm trying to get the widget working perfectly before I dive into fixing that, as a split text widget can work perfectly fine on its own, but a perfectly integrated lousy widget is pretty worthless.

I tried a new time management method today, called The Pomodoro Technique. It worked very well for me and I got in more completely focused, uninterrupted coding time today then I ever have, and really hit a few issues hard, which makes me very happy. Hopefully my head will adjust to work at that pace and it will get less exhausting.

In some other news, I talked to Greg and we discussed how I went about learning how IDLE works and teaching programming in general. In specific, we discussed the apparent contradiction between my needing to modify IDLE and build little things with Tkinter to gain understanding of them (my friends are getting less amused by joke GUIs, my girlfriend is not) and his ability to just read through source code and understand how it works, which I covered here. I don't think these two approaches actually contradict. Though I understand the base language, this is my first experience with GUI toolkits or large projects with codebases stretching back over a decade. I believe, like in math, the only way to learn any programming concept is to use it thoroughly. Programming is playing with invisible abstractions, and using them changes the way you think. It's not possible to understand them by just reading until those changes have already been made and you have an intuitive understanding of the abstractions. Greg has put in his time and done some major coding, so he has a larger and more complete framework already in place to deal with these abstractions in a way that I do not. However, the only way to get an understanding like his is to program until it is intuitive, to work until the right thing is obvious (or at least visible) and nonsense paths simply aren't thought of. So needing to do to learn isn't in contradiction with simply reading through; the former is the path to the latter. Hopefully, what I am working on will make those first steps of doing a little less painful, and lead me to understand how this all fits together.

Chasing Rodents

2010-06-15T20:30:00.002-07:00

I feel like today followed my plan the closest of any I have had so far. I hashed out the final details of my new split Text widget and the testbed for it. Right now the widget behaves just like the standard Tkinter.Text, but with two of them. Tomorrow morning I should be able to get it working completely. Today I made a very fun test program to test out my focus follow method, which my girlfriend likes a lot. Here is a picture:

It says 'Hi, Lily' when it gets focus and 'Bye, Lily!' when it loses focus. You can probably guess which text the "Who's awesome?" button is responsible for. Overall, today was quite a success and I look forward to talking to my mentor and getting the basic prototype finished tomorrow.

Today In Reading Documentation

2010-06-14T19:59:00.000-07:00

Today was very productive! I finished figuring out the function and finer points of all the methods required to simulate the Text widget for IDLE. (Although I did spend an hour chasing down documentation and reading source code to find out what a method did, only to find it was a typo and didn't really exist.) I got a lot of work done on my test fixture (the text fixture is to make sure that Tkinter.Text and my Text are interchangeable without dealing with the complications of how they are used in IDLE.) I don't have anything working well enough to show the world yet, but it should be pretty close to what I need by the end of tomorrow. Tkinter is a little awkward to work with--it shows its tcl roots more than I would like--but I now thoroughly understand how the code that deals with it works, so if all goes well tomorrow, I should be able to drop my replacement widget right into some slightly tweaked editor classes and be on my way!

Hey Buddy, Can You Think of a Way to Sort This Really Slowly? What? Way Too Slow.

2010-06-13T07:59:00.002-07:00

Fan of CompSci as a humor discipline?
My friend gives us a shout out.

I Heart OOP

2010-06-11T11:25:00.001-07:00

I'm working on the newest plan to make my prototype work and I'm very very grateful right now that python is duck typed and polymorphic because I'm approaching this a little sideways, but I think it is the best way to do it. Back to work now. I should have a pretty massive Google Code hg check-in by Monday, but it will probably look quiet on this front until then. Also, if you make a cute shell script to work through a directory and pull out all of the chunks of text with a certain phrase in them, don't run the script twice without deleting the output file. You will be really confused.

Book Review: Python Essential Reference

2010-06-09T21:30:00.003-07:00

Now that I have a reasonably sized Python project, I thought it was time to pony up for some real dead-tree documentation for the language, rather than trusty /usr/share/doc/python2.6/html or the Python website. I don't know about you, but I find it kind of difficult to use my thumb to mark a page in w3m and find it easier to flip through a real live book. So, I would definitely recommend buying something to keep on hand if you want some help remembering all of the Python standard library and language features and how to use them. I got Python Essential Reference by David M. Beazley to fill this role for me.

If you already know Python, it is a very lucid summary of the widely used aspects of the language. For me, it clarified many details of functional Python and exactly how Python treats object orientation. However, I did need to consult outside documentation about coroutines as I hadn't been exposed to them before. For most uses, Python Essential Reference is a very thorough explanation of what you would need of Python. However, I was disappointed when I needed it as a reference instead of just a language refresher, as the first two standard library modules I looked up (code and linecache) weren't featured so I needed to look them up in the Python Online Docs anyways. Thankfully those are both very simple modules and I grokked their usage quickly, so, though their absences are marks against the book, they weren't problematic for me.

If you already know one language well, you could probably learn Python from this book, but I would recommend Dive Into Python instead; it is very good. If you don't know how to program, I would recommend starting with Python, but not from this. The Non-Programmer's Tutorial for Python 2.6 would be a better choice. Overall, I have found Python Essential Reference to be a useful reference guide and would recommend it if you are looking for a quick, comfortable reference guide, but don't need something definitive.

Finally Writing

2010-06-08T00:07:00.006-07:00

In Code Complete it says professional software developers write an average of 250 lines of code per day. When I first read this I scoffed, thinking that was incredibly low for ostensibly talented programmers. Today, I was envious of that average. What I hadn't taken into account in the past was the challenge of integrating new work with legacy code. Previously, I have been making code from scratch or with the original authors in person so I had a clear idea of where things needed to go from the start. Today, I was working with code first written in 1998. I wrote five times as much code as I ended up committing, after going down one path, looking at the way the existing architecture was and realizing I was doing it a completely untenable way. However, this led me to see how the front-end fits together in a comprehensive way for the first time, though the RPC mechanism tying it all together remains cryptic. The feeling of working with the code, playing with it, and knowing where to look when I don't understand something instead of just bashing the codebase and reading library documentation for hours is really exciting. Knowing how things work is more fun than recognizing names. (I should really post a comprehensive architecture analysis when I am done with the prototype and planning the final integration, so the next people working on IDLE will have an easier time getting up to speed.) By the end of the day, I found out exactly what direction my modifications need to take to have everything working out. I really just wanted to make a night of it, but I'm moving to Baltimore tomorrow, and packing took precedence. For the first time on this project, I am feeling confident; I'm looking forward to really getting this working.

Someone Stole My Teeth

2010-06-03T19:39:00.000-07:00

I didn't do much work today. Instead, I got my wisdom teeth out; since I am programming in Python, not chipmunk, I took my mentor's advise and watched bad television, drank milkshakes, and slept.

Code is starting

2010-06-02T13:26:00.000-07:00

I haven't checked in anything but the original IDLE code yet, but I have project hosting for the testbed I am building for various markup schemes. Here it is.

It's Easter!

2010-06-01T16:53:00.000-07:00

On my way to the idlelib directory in Python 3's source code, I found an amazing Easter Egg. In Python 3, if you type 'import antigravity', it works. The module contains an implementation of Randall Munroe's geohashing algorithm, which makes me ludicrously happy. Python is the best.

If you don't get the joke.

My future profession.

2010-06-01T01:04:00.007-07:00

I fixed it.

Broken Version from Greg Wilson
by Cowbirds in Love
Today, I worked through a Tkinter tutorial and read IDLE code so I don't really have a lot to report unless someone wants to seem some really ugly GUIs. It seems like there is a better argument for "comment style" annotations so that design should be done soon and I can start building a parser/tracer and I won't feel like I am in design limbo quite as much.

Annotation Design Examples

2010-05-28T21:58:00.003-07:00

These are my designs for the command language to control the IDLE extension I'm working on. For people who haven't been to this blog before, I'm working on a program which will, in conjunction with IDLE (the Python IDE), interactively display and explain code for use in Python tutorials. The basic idea is that it reads specially marked up files and uses the markup to explain the code as it is executed. This way, a student can open one of the files in her IDE and work through a tutorial interactively, much like they way she would interact with a teacher. These are my designs for the command language. I would love to hear your feedback, especially if you are currently involved in teaching Python or creating online tutorials.

This first one takes the same approach as pdb, embedding the control functions along with the code being explained.


#!/usr/bin/env python3
#This uses a single library that is interspersed and controls the program
#This could not be executed without using the library

import teacher

teacher.annotations_on=True

teacher.explanation("This can be thought of as x becomes 5")
x=5

v=input("please enter something: ")

teacher.explanation("This is a list")
y=[4, 1, 5, 'string']

teacher.explanation("This prints the string and the value of v to the screen")
print('this is example output to the screen', v)

i=0

 teacher.eachtime("this is time %d through the while loop, the value of x is %d" % i, x)
while True:
     if x == 8:
                break
        else:
             x+=1
             i+=1

teacher.oncall("The previously defined code is being used now")
def function(x):
       local=x
      return x*2

teacher.thistime(0, "this starts with the first item in the list and works through all of them")
for item in y:
        print(function(item))

This program can't run without the external library, although setting the annotations_on constant to False could make it run as though all of the library functions were commented out. The next idea comes from a very different train of thought, identifying the line numbers to comment on and leaving the actual source code untouched.

Here is the clean code with line numbers for reference:


1 #!/usr/bin/env python3
2 #This is a piece of example code to demonstrate several techniques for controlling presentation
3 
4 x=5
5 v=input("please enter something: ")
6 y=[4, 1, 5, 'string']
7 
8 print('this is example output to the screen', v)
9 
10 while True:
11         if x == 8:
12                  break
13         else:
14             x+=1
15 
16 def function(x):
17         local=x
18         return x*2
19 
20 for item in y:
21         print(function(item))

And here is the command file:


#direct line number annotations

4: This can be thought of as x becomes 5

6: This is a list

8: This prints the string and the value of v to the screen

10 for_all: this is time %d through the while loop, the value of x is %d % i, x

11: the previously defined code is being used now

20 first_time: this starts with the first item in the list and works through all of them

This idea does away with the external library or use of regular python code, instead using a custom tagging format. For extensibility, it would be necessary to develop a way for users to define their own tags.



#!/usr/bin/env python3
#These tags would be known by the interpretter

$NOTE "This can be thought of as x becomes 5"
x=5

v=input("please enter something: ")

$NOTE "This is a list"
y=[4, 1, 5, 'string']

$NOTE "This prints the string and the value of v to the screen"
print('this is example output to the screen', v)

i=0
$FORALL "this is time %d through the while loop, the value of x is %d" % i, x
while True:
       if x == 8:
                break
        else:
                 x+=1
                i+=1

$ONCALL "The previously defined code is being used now"
def function(x):
       local=x
       return x*2

$FIRST_TIME "this starts with the first item in the list and works through all of them"
for item in y:
      print(function(item))

This version tries to clean out some of the extra code, moving it instead to an external file for definitions.


#!/usr/bin/env python3
# tags defined in an external file

$variable_assignment 
x=5

v=input("please enter something: ")

$first_list 
y=[4, 1, 5, 'string']

$print 
print('this is example output to the screen', v)

i=0
$while_loop(scope='all') 
while True:
        if x == 8:
                break
        else:
                 x+=1
                 i+=1

$on_call(scope='call')
def function(x):
       local=x
        return x*2

$first_time(scope='first')
for item in y:
        print(function(item))

Example external file:


variable_assignment: "This can be thought of as x becomes 5"
first_list: "This is a list" 
print: "This prints the string and the value of v to the screen"
while_loop: "this is time %d through the while loop, the value of x is %d" % i, x
on_call: "The previously defined code is being used now"
first_time:  "this starts with the first item in the list and works through all of them"

These are representative of the four main approaches I have thought of. However, there are a lot of possible variations among them. For instance, the symbols at the start of the tags could be replaced by something like #> which is notable, but would register as a comment in normal usage. An external tag file could contain custom python functions, perhaps using a custom library like the first example. I would love to hear what sort of language you would like to use, if I completely missed something, or hit something on the head. I tried to demonstrate how some basic annotations would work, but I glossed over advanced features so if there is something you would like, please leave a comment.

What I Thought I Would Be Doing Today, and Where I Am Now

2010-05-28T02:44:00.001-07:00

Today I thought I would be reading a few papers and working through several annotation examples to be posted up here. I also thought I had a generally stable view of what sort of thing the software I am making would be required to do. I was wrong.

I've been looking into scriptable debuggers, trying to glean some insights into the markup I am working on by seeing how they solved the problems I am facing. I did gain insights, however, by thinking of what I am working on in the light of debugging, the problem-space has opened up tremendously regarding expectations for the language. Thankfully, what I am working on is not as daunting as scripting a debugger--this project makes explanations, not experiments. The requirements driving its relation to external code are essentially the same though: it needs to observe the state of the program, monitor its control path, and control the execution so looking at debuggers (and how they are automated) is a great starting point.

I found a paper called "The design and implementation of a dataflow language for scriptable debugging" by Guillaume Marceau et al., which was very insightful into the core problems of debuggers and scripting them. Although I don't think its functional approach is appropriate for the application I am working on, it brought to light some truth about the problems I am approaching. First, the authors noticed that this class of problems is inherently event-driven; the scripts themselves are "passive agents". Second, the language must have broad capabilities (or be extensible) as it is not possible to predict how the user will want to use the language. In this sense, a scripting language for a debugger (or a control language for a teaching tool) can be very close to a general purpose language.

This led me back to something I was previously thinking: why can't I just embed Python as my language? It's a great scripting language. This train of thought was bolstered by my research into GDB's scripting capabilities. It uses Python! I'm not sure how much of what Python does will need to be adapted (or if it just needs a custom package) for my usage, but this is certainly a promising trail.

Tomorrow, I'm going to track down the web of citations surrounding the Marceau et al. paper and the debuggers they directly reference and see where they went with those ideas, figure out exactly what in depth scripting of gdb with Python looks like, and crack into pdb's source to look at how event-driven Python looks. I'm really excited to see where this is going, but it really drove my plans off the rails regarding what I expected the markup to look like and how I expected it to work, but it opens up some really interesting ideas and approaches. Barring any major revelations tomorrow, I will have markup idea examples up for feedback from the potential users by the end of the night.

On Code Reading

2010-05-27T04:09:00.005-07:00

Today I had my first conversation with my mentor for GSOC, Greg Wilson, and at the end of it I asked for some advice about reading code, as I feel it is a major weakness of mine, having a hard time figuring out code written for a problem I have not already solved. He had a very insightful response, bringing up how much computer education is about writing and reading is mostly ignored. (Imagine an English education like that!) He then referenced the book Code Reading (really the only one on the subject) and a walkthrough of his process which he wrote on the subject for a software engineering class he previously taught.

Before I get into what I took from that piece, I think a brief overview of my previous code reading process is in order. I basically tried to read it like a book (or a piece of code I had previously written) starting at the top and reading it through to the bottom, with a shell and a browser open, to read external documentation and work out how any unfamiliar libraries or built-in functions operate. After I feel like I have a reasonable idea of the architecture, depending on how well I need to understand it, I will run it through a debugger to see the control flow in depth and see how it works. With this comes tinkering with the code to get a feel for how it is interconnected.

Greg's approach starts the same as my old one (by reading any explanations at the top), however, I was surprised that he didn't even look at any code until he found the start point of the program, in this case the if __name__ == '__main__' idiom, and following the actual execution from there, diving in and seeking to understand the code, only as it was naturally used. This makes a lot of sense as code can be quite meaningless without context and it is very easy to lose track of the important details, trying to keep everything straight. This way the details sort themselves out a little better. Another surprise for me was his willingness to contact the original authors of the code to ask for clarification, which makes perfect sense--they know it the best--but for most of the code I am reading, the original authors seem very far away and not welcoming to questions about how their software works, so I never thought to contact them for clarifications. This seems like a foolish assumption to write out, so it is good to see an established CS professor contacting the original authors, making it seem less intimidating to me.

I was happy to find another person sharing my problems and surprised how long it took Greg to read the 1800 lines of code (3 hours and 20 minutes) and the number of breaks he took in the process. I have always tried to take it all in one big gulp and been disappointed to feel somewhat lost after my first look over, now I know this is a normal state of affairs.

Though I was surprised by many of the things I hadn't thought of, I was relieved to see I was on the right path. We both searched for unknown functions and variables as they were used and referred to their definitions frequently while reading. I was also happy to see I wasn't the only one with a tendency to rewrite the code in my head to help it fit with my programming quirks.

Although it is probably not practical for larger projects, I am going to adopt Greg's practice of physically printing out the code in question, making it easy to reference and annotate, as I come across more unknown code in the future. If anyone has tips about reading code, I would love to hear them in the comments.

Thanks Google!

2010-04-26T22:18:00.000-07:00

I got into the GSoC and am incredibly excited and more than a little bit nervous. This is the largest project I have ever attempted and I really want to see it become a reality. I am really thankful for the help Greg Wilson has given me so far and look forward to having him as my mentor this summer. Many more updates to come.

Playing It Safe Will Always End In Disaster

2010-04-08T22:32:00.000-07:00

This is the future home of my summer project documentation. Right now, I'm applying for the Google Summer of Code to help extend IDLE, the Python IDE, so it can be more interactive in conjunction with online (and offline) instruction. I have some interesting things nearing completion and looking for a home; so, no matter what this space is used for, things are coming.

Now, that I have that out of the way, a little bit about myself. Sometimes, I make calculators out of dominoes and succeed. Sometimes I make hovercrafts out of old fans and fail. Miserably. Sometimes, I just destroy everything I can get my hands on for the sake of making something more. I am occasionally known to break household objects with little provocation and less of an excuse. I find thought-terminating cliches terrifying and linked structures fascinating. I believe that in a world of people hiding behind their fear, risk-takers will always have a place.