Monday, December 27, 2010

In order to access a Garmin device from Javascript in a web page, the Prototype framework needs to be loaded. The bad thing about Prototype is that messes other things up. One nice thing about jQuery is that it is nonintrusive, defining only one global symbol. Prototype, on the other hand screws with basic Javascript datatypes.

The first thing that I had to deal with after loading Prototype was that I could no longer use for (i in array) { ... }, to iterate through array elements without getting a bunch of extra junk. So I rewrote my for loops to go from 0 to array.length - 1 instead.

The next thing really confused me for hours. I wanted to encode a Javascript data structure as JSON, and JSON.stringify([]) would inexplicably result in "\"[]\"". But if I used the browser's Javascript console, JSON.stringify([]) would correctly result in "[]". Finally, I came across a web page that identified Prototype as causing this problem by setting Array.prototype.toJSON, among other things, which also explained to me why Prototype was screwing up my for loops iterating through arrays.

Monday, December 20, 2010

When building a war file with ant, I found it convenient to include multiple <webinf/> elements in the war task when some of the WEB-INF files were generated, so that the source files and the generated files could be in separate directory trees. However, if there were entries under WEB-INF that were duplicated between the directory trees, one of them would always cause ant to find the war file out of date and rebuild it. It's not that I had any duplicated regular files, but I did have duplicated subdirectories, which had different timestamps. I had been puzzled by why ant was always rebuilding the war file, even if nothing had changed, before looking into it. I finally changed the war task to have a single <webinf/> element pointing at the generated files, and added a task that copies the source files into the directory tree of the generated files.

Monday, December 13, 2010

Facebook authorization has some weirdness that I could only deal with with an ugly workaround. My understanding of OAuth is that I redirect the user to a Facebook URL. Then, Facebook sends the user back to me with a code that I exchange for an access token with Facebook. This works fine if the user has already authorized my application: the user gets redirected to Facebook, and then redirected back to me without a hitch.

However, if Facebook need the user to authorize my application, then there is a problem. My application runs in an iframe in a Facebook page. However, if I redirect the user to Facebook, which shows the user the authorization page, Facebook detects that it's in an iframe and grays out the page. If the user tries to interact with it, the user gets sent to the page out of the iframe. Once the user authorizes my application, the user gets sent back to me, but not in the iframe. Fortunately, when the user comes to my application, Facebook passes a flag to me that says whether the user has already authorized my application. So if the user has already authorized my application, I redirect to Facebook and things work fine. If the user has not, then I show the user a page that has some javascript that sends the user to Facebook out of the iframe, and when the user comes back from Facebook, I redirect the user to the Facebook page that frames my application.

Monday, December 6, 2010

I have an ant target that runs some scripts on some Amazon EC2 instances. So, I first generate a file with a list the hostnames of the running instances. Then, ant runs a shell command that iterates over the hostnames and runs ssh to each with a command. Since I didn't want to run the command as root, the command was su -c "\"command with some parameters\"" username, which worked fine for me. However, on cygwin on Microsoft Windows, the nested quoting didn't work, so that su was trying to set the user to the first command parameter rather than the username I specified. None of the variations of single and double quotes and backslash quoting the space characters worked.

Finally, I gave up on trying to deal with the quirks of the Bourne shell under Microsoft Windows, and moved the su into a script on EC2:

if [ ! `id -un` = username ]; then su -c "sh -c \"$0 $*\"" username; exit; fi

Of course, none of the command parameters are expected to have spaces in them, as the $* would be trouble. And $@ wouldn't help because of the nested quoting.

Monday, November 29, 2010

For storing objects in a database, I chose to serialize the objects in a way that maintains compatibility between object versions. So Java serialization was out. My first implementation was to serialize to JSON using Jackson, which was simple because it could serialize and deserialize objects that I was already using. However, there were more compact serialization schemes. Apache Thrift and Google Protocol Buffers looked fairly equivalent in functionality. Both required generating code from some IDL, making them less convenient than JSON. I chose Protocol Buffers since its serialization seemed to be slightly smaller than that of Thrift.

In order to continue using the objects that I was using before, since they included annotations and logic that can't be cleanly added to the protobuf-generated code, I used java.beans.Introspector to extract values from the objects to be stored in the database and put them into the protobuf objects and vice-versa.

Monday, November 22, 2010

AWS (Amazon Web Services) is pretty nice. It not only has what's needed to deploy and scale a web service, but everything can be controlled through their REST (or SOAP) API, so that everything can be automated.

One thing that should have been obvious was how a service would start when an instance starts. The first time, I created an instance from a prebuilt image, copied the code to it, then logged in and started it manually. Eventually, I created an image that included all the code. I finally figured out that I could start it in /etc/rc.local.

The next thing I needed to figure out was how to update the software. Everything would be in a single war file (except for static assets pushed up to the CloudFront CDN (Content Delivery Network)), but automatically building a new image from an existing image with the war file replaced looked difficult. I could do it manually by starting an instance, replacing the war file, and then creating a new image. I could automate that manual process. Another way seems to involve turning the image into a loopback filesystem, replacing the war file in the filesystem, then turning the filesystem back into an image, then uploading the image. Finally, I figured out that I could just upload the war file to Amazon S3 (Simple Storage Service), and the instance could download the war file from S3 when it starts up, and creating new images is unnecessary.

However, the process of getting a working image was slow and tedious, since starting and stopping instances, and creating images were all really slow. Once I had an image that worked, setting up load balancing was trivially easy. Setting up Auto Scaling also looks very easy, once I figure out what metrics to use for launching and terminating instances.

Monday, November 15, 2010

JAX-RS is really nice for writing REST services. I'm using Jersey 1.3, with Guice 2.0 and jersey-guice to configure things. However, the Jersey JSON provider is horrible. It messes up writing empty and single element arrays, and it writes boolean and numerical values as strings. My first hack around that was to return a JSON string instead of the objects, doing the conversions in-line using Jackson 1.5. That was ugly, so I overwrote META-INF/services/javax.ws.rs.ext.MessageBodyReader and META-INF/services/javax.ws.rs.ext.MessageBodyWriter in the Jersey jar, replacing the Jersey JSON providers with org.codehaus.jackson.jaxrs.JacksonJaxbJsonProvider, and it was much better.

I also use the Jersey REST client to talk to Facebook. However, Facebook returns text/javascript;charset=UTF-8 as the content-type, which is not recognized by either the Jersey JSON provider, or the Jackson JAX-RS JSON provider. Again, the first thing I did was to get the content as an InputStream, which I sent to Jackson for object binding. Then, I figured that I could just extend the Jackson JSON provider to accept text/javascript and use that class as the JSON provider.

@Provider
@Consumes({MediaType.APPLICATION_JSON, "text/json", "text/javascript"})
@Produces({MediaType.APPLICATION_JSON, "text/json"})
public class MyJaxbJsonProvider extends JacksonJaxbJsonProvider {
@Override
protected boolean isJsonType(MediaType mediaType) {
if (mediaType != null && "javascript".equals(mediaType.getSubtype()))
return true;
return super.isJsonType(mediaType);
}
}

Now, the code no longer has ugly hacks for JSON. However, any time I get a new Jersey jar, I'll have to overwrite the provider list in the jar, which is ugly. Perhaps I could put my own javax.ws.rs.ext.MessageBodyReader and javax.ws.rs.ext.MessageBodyWriter in a jar, and have it take precedence, but it didn't work the first time I tried it. It might depend on the classpath order of the jars, and having to use the classpath order of jar files to override things is also ugly.

Also on JSON, RFC-4627 says that slashes may be backslash quoted, though the example in section 8 does not backslash quote the slashes. The JSON from Facebook does backslash quote the slashes, but the JSON produced by Jackson does not. I wonder why backslash quoted slashes were explicitly included in the JSON specification. Perhaps it was inherited from Javascript, in which case I wonder why it was that way in Javascript. My only guess is that it had something to do with regular expressions, if it was from Javascript. Otherwise, it just seems weird.

Monday, November 8, 2010

One thing that bugs me about some of the datacenter hosts that I have to access for one reason or another under a generic developer account that has no write access to anything except /tmp is that there is a long delay while xauth tries to lock $HOME/.Xauthority. I've explicitly unset DISPLAY, but that makes no difference.

Monday, November 1, 2010

My initial attempt at making an iframe Facebook app immediately ran into a snag. It seems that if the Safari browser's cookie configuration is "Only from sites I visit", subtitled "Block cookies from third parties and advertisers.", then cookies don't get saved or sent to the iframe. Which means saving the session id in a cookie fails. Having sessions is critical, but I don't know if other browsers have this problem, or whether it's acceptable to fail for users that block iframe cookies (or all cookies). The session id could be stuck in the URL as a matrix parameter with a simple configuration of the servlet container, but it's painful to have to make sure all the URLs that need the session id are rewritten. Plus, the session id would get leaked in Referer headers.

Once I made sure all the URLs had the proper rewriting, it still didn't work, because although Glassfish v3 (74.2) rewrites all the URLs, it never recognizes the sessionid in the URL, and creates a new session. When I switched to Tomcat, it worked with cookies disabled.

Monday, October 25, 2010

Now, I'm working on Facebook apps using OAuth 2.0, and the draft specification says in section 4.2 that JSON is returned when retrieving the access token. However, Facebook's OAuth 2.0 server returns application/x-www-form-urlencoded data. I anticipate things changing without notice, so I'll have to code it flexibly. Of course, it is a draft specification.

As for OAuth 2.0 versus OAuth 1.0, it's a little nicer. Code for computing the signature for OAuth 1.0 is difficult to debug, but, once it's working, it's easy. On the server side, incorrect nonce handling seems like an obvious hole for replay attacks, and OAuth 2.0 is better in that way. Also, unencrypted OAuth 1.0 traffic can still be snooped. Of course, OAuth 2.0 is still subject to traffic analysis. Of course, with Facebook's reputation on privacy, these differences are irrelevant for most Facebook apps, including the ones I'm working on.

Monday, October 18, 2010

After using Guice for a little while, I've started learning how to use it better. I can use @Provides methods to configure objects. I like being able to install() modules for modularity.

For webapps, I can have two web.xml files, one pointing to one configuration, and another pointing to another configuration, such as a test configuration, by specifying the different configuration classes as the listener in web.xml files, where the bindings specify different implementations.

I'm not sure if I like the fluent interface. It's nice for reading the code, but it makes referring to the javadoc more difficult. It also creates a bunch temporary objects, but that isn't really a concern. Guice is supposedly a lot faster than Spring, and the temporary objects probably can be optimized away by escape analysis.

Monday, October 11, 2010

I've only had a cursory look at Google Guice, but this is my first impression as compared with Spring, which I've used a lot more. Using Spring has actually changed the way I approach writing code.

What I like about Guice is that it is statically type-checked. Spring's xml configuration is like a dynamically typed language, and more errors slip through to later stages.

What I like about Spring is that code doesn't need to be written with any Spring-specific calls or constructs, whereas Guice requires Guice-specific annotations. If I only had to deal with code I've written, it wouldn't matter so much. However, Spring makes it hooking in code from external libraries more convenient.

Also, when using Spring, I like being able to specify every last configurable flag in the xml configuration, and anything that truly needs to be tuned can further be extracted into a properties file. Guice seems to make that level of configuration more inconvenient. Perhaps I am overlooking some mechanism, but it seems inconvenient to have to insert yet another annotation to name yet another string field that I want to configure, rather than just being able to inject beans. Guice seems to encourage having a few, high-level, injected objects, rather than every last configurable value injected.

One nice thing about Guice is that it can inject private fields, while Spring requires a public setter method. This is just a minor thing though, since, while the implementing class may have some public setters, the public interface that it is being injected as won't have those public setters. There is also constructor injection, but that is not convenient when there are many parameters to inject.

Monday, October 4, 2010

I ran across a bug in an Android app that I was working on that had me throwing the phone to the floor multiple times in frustration. I finally tracked the problem down to SurfaceHolder.Callback.surfaceChanged() being called twice when the phone is held vertically, but only once when the phone is held sideways. Everyone using the app holds the phone sideways when using the app because the text and icons are all sideways. However, I had said to myself, "Screw that. I'm not going to turn the phone sideways just for that." I don't know what the proper way to fix it is, since I don't know why the code is in SurfaceHolder.Callback.surfaceChanged(), and the person who wrote it had just moved on to another company (Google, actually) a week earlier.

Monday, September 27, 2010

Here's a bug that I recently came across. I didn't figure it out, and it wasn't code I wrote, but it is an example of gratuitously bad code.

Originally, I had written some code for a quick-and-dirty demo that went something like:
    packetType = readPacket(input, buffer);
switch (packetType) {
case TYPE1: parseType1(buffer); break;
case TYPE2: parseType2(buffer); break;
...
}

where readPacket() read the appropriate number of bytes into the buffer depending on the packet type. Something like:
   packetType = input.read();
if (packetType == TYPE1) {
read some stuff into buffer;
} else {
count = input.read();
read count bytes into buffer;
}
return packetType;


Somewhere along the line, in a change in source control that was something along the lines of "update to the latest protocol", there were a number of changes, including gratuitously changing the code from something like the above, to something like:
    packetType = readPacket(input, buffer);
switch (packetType) {
case TYPE1: read more stuff from input; break;
case TYPE2: parseType2(buffer); break;
...
}

and readPacket() was changed to
    packetType = input.read();
if (packetType == TYPE1) {
return packetType;
} else {
count = input.read();
read count bytes into buffer;
}
return packetType;

which all would have worked fine, even though it was a stupid and gratuitous change, except that there was another place that called readPacket(), but ignored the packet. So, if that other place called readPacket() and got TYPE1, the packet wouldn't be fully read, and subsequent calls to readPacket() would return garbage. I don't know what the thinking was behind changing the code like that.

Monday, September 20, 2010

One component of this demo that I'm working on is a server in C++. The guy who did the work on it was laid off 5 months ago, and now I have to add crap to it. Most of it was 3rd-party code. I don't know what changes have been made to it, since it was checked into source control in one big chunk 9 months ago. However, there is one directory that has our specific configuration files and build and startup and shutdown scripts.

I found the build, startup, and shutdown scripts kind of annoying. They all assumed that the scripts would be run from the directory that the script were in, which I don't like to do. I prefer running /absolute/path/to/the/script, rather than cd /absolute/path/to/the, then ./script. So I added cd `dirname $0` to the top of all the scripts. Another goofy thing in the scripts were the use of pushd somedirectory >/dev/null, and then popd >/dev/null. Personally, I use directory stacks interactively all the time, and often have 3 or 4 directories that I switch between, but, for the script, cd somedirectory, then cd .. would work just as well.

As for the C++ server, I had to add a feature, which I did. And it worked some of the time, but mysteriously didn't work some of the time. There was a logging framework that I tried to use, and I spent a day trying to use it. When I tried making a new logger class, and tried enabling it, the server crashed. Finally, I just hacked in some code that fopened a file, and vfprintfed crap directly to it, and figured out that I had added an uninitialized variable to a class, and, depending on what value it had, the feature worked or didn't. So I fixed it by initializing it in the constructor.

Monday, September 13, 2010

There is some demo code at work that's been around for around 10 months, and I've added various features to it from time to time. Also, from time to time, the deployed code has to be put in a "known state" to actually do the demos. A month or two ago, some other guy added a feature, which makes the user interface fancier, so his changes need to be in all the demos, but he never checked anything into source control. Since then, I've added some features, all of them checked in. So now when there is a demo, this guy has to merge my changes into his code and then deploy it on the demo system. Except that what has actually happened is that they go back to the code that he originally deployed over 3 months ago, which doesn't have any the features I've added, which include changes that make the platform more robust. I can't deploy the demo including his features because I don't have his code.

One of these days, I'm going to ask him why he doesn't check his crap in.

I don't understand how some people can work on stuff while rarely interacting source control. I update my source trees often. I update before starting work. I merge in changes periodically during work. I merge in changes when I'm ready to check stuff in. I check stuff in when I'm done working on it.

Monday, September 6, 2010

There has been a bunch of attrition at work, so I have to help taking over some Android code. So, to learn the Android platform, I decided to make a sample application that receives incoming SMS messages and send them through the text-to-speech processor. I put the code on github. I managed to get it to work on an emulator in a couple of days, as the Android platform is pretty easy to learn and pretty well documented. I haven't tried it on a real phone, since I wrote it for Android 2.1, and phones at work run Android 1.6.

The one thing that I had problems with was the service lifecycle interfering with the asynchronous text-to-speech interface. My ugly solution was to have the service post an empty intent to itself while the text-to-speech was still speaking to keep the service alive, since the text-to-speech instance would have been considered leaked if it isn't shutdown when the service is destroyed. If I continue working on Android applications, I'll probably find a better way to do it.

Monday, August 30, 2010

I just started using git for version control, and made some free repositories on github. I haven't tried anything beyond the very basics of creating a repository and committing and fetching code from multiple sites. I haven't made any branches or done any rebasing. For the basics, the only new concept I had to learn was the index, which I glossed over when first reading the documentation. After playing around a bit, I got a little confused, then kind of figured out what was going on, then read the documentation again, and it became clear.

Monday, August 23, 2010

More Project Euler problems.

For problem 51, I only had to try replacing the 0s, 1s and 2s, since there had to be 8 values in the family. And I didn't have to try replacing 1s with 0s or 2s with 1s or 0s, since those would be redundant.

I just did a brute force search from 1 on up for problem 52. It would be more complicated and maybe faster to skip numbers where the most significant digit is not 1, but the answer wasn't all that big, so it wouldn't be significantly faster, if at all.

For problem 53, since nC0 = nCn = 1, and C increases rapidly from the edges and peaks at the middle, I counted the values under a million for 0 ≤ r ≤ n/2, doubled it and subtracted.

For problem 54, using Haskell's deriving (Ord) with an appropriate data structure and mapping the card values to values that ordered the ace, face cards, and ten correctly made it simple. Also, there were no instances of A 2 3 4 5 in the data, which, if considered a straight, would have had to have a special-case constructor to be ordered correctly.

I just brute-forced problem 55 over the range 1..9999.

For problem 56, the upper bound for the sum of digits is 9b log10a, and log10a < 2, so the upper bound is 18b. So the maximum value for b = 99 establishes a lower bound for b that needs to be searched. Subsequent new maximums for b < 99 can raise that lower bound. I wound up searching 1 ≤ a ≤ 99 for 54 ≤ b ≤ 99.

For problem 57, I found that doing head (show numerator) < head (show denominator) was faster than counting all the digits, as it's known that numerator/denominator is approximately 1.4.

For problem 58, I just did a brute force iteration over each square, with running counts of the total and the number of primes.

For problem 59, I printed out every 3rd character decoded with each of the 26 possible key characters, and then visually chose each of the 3 characters in the key.

For problem 60, I could only come up with a super slow solution, though it did come up with the answer in about 10 minutes, and ruled out other possible answers after 30 more minutes. I made a very wide, shallow tree, spitting out results when reaching a depth of 5. I had to do a bunch of redundant calculations, because although I had a branch starting with 3,7,109,... I still needed branches starting with 3,..., 3,7,..., 3,109,..., 7,..., 7,109,..., 109,.... I'm thinking that a graph-based algorithm, instead of a tree-based algorithm would be faster, should I revisit this problem.

Monday, August 16, 2010

Looking at more Project Euler problems.

I thought problem 41 was going to be tough, but there are only 9! = 362880 nine-digit numbers to consider, and even fewer 8-digit numbers to consider, etc, so it was just a matter of checking for the first prime permutation.

Problem 42 was straightforward. I made a list of triangle numbers up to the maximum word value, then filtered the word values and counted.

For problem 43, I started by taking all multiples of 17 under 1000, filtering out any with repeated digits. Adding the 4th digit made the set 7 times bigger, but removing those that did not have the divisible by 13 property made the set about half the size. Iterate for the remaining digits.

I couldn't find a reasonable way to solve problem 44. It involves finding 4 pentagonal numbers, pi, pj, pk, pl, where pi ≤ pj < pk < pl, and pi = pk - pj, and pj + pk = pl, where pi is minimized. For a given i, there's an upper bound for j, such that pi ≥ pj+1 - pj, and the upper bound for j is O(i2), which means a brute-force search over i gets extremely slow. However, since I didn't have any better ideas, I searched i from 1 to about 350 or so without finding a solution, and gave up.

For problem 45, I made a list of hexagonal numbers, filtered out those that weren't pentagonal or triangle, and took the 3rd number in the resulting list, the first two being 1 and 40755. It would have been slightly slower to make a list of pentagonal or triangle numbers, and filter out those that weren't the other types.

For problem 46, I iterated over non-prime odd numbers, building up a list of primes and doubled squares less than it, stopping when none of the primes could be added to one of the doubled squares to get that number.

In problem 47, I only had to check every 4th number until one had at least 4 factors, then check the previous one, the one before that, and the one before that. If any didn't have at least 4 factors, I could jump forward 4 from that.

For problem 48, using mod 1010 arithmetic means not having to add gigantic numbers, while still getting the last 10 digits.

For problem 49, I took all the 4 digit primes and grouped the ones that were permutations of each other together, then searched each group for a qualifying triplet.

For problem 50, I found the longest prime sum starting with 2, as well as the list of primes after the last term of the sum. Then, I subtracted 2 and added the next largest prime, looking for the longest sum starting with 3, and iterating until subtracting the next smallest prime and adding enough of the next largest primes to make the sum have more terms than the current longest sum resulted in a sum more than a million.

Monday, August 9, 2010

Looking at the 4th ten Project Euler problems.

Problem 31 is a straightforward recursive count. Once you get to the 1p, the number of combinations is 1.

For problem 32, 99×99 = 9801 and 111×111 = 12321, so all the solutions have to be a two-digit number × a three-digit number resulting in a four-digit number. I did a brute-force search through all the combinations of 5 digits of the multiplicands.

Problem 33 just involved searching with 1 through 9 to see if any pair of digits had the property, so that was a small search. Also, there were only 36 pairs of digits to check.

Problem 34 is just like problem 30 with a larger bound.

For problem 35, I partitioned the primes by number of digits, then filtered out the numbers that were not the minimum of the digit rotations, which conveniently eliminated all numbers with zeros. I then filtered out the numbers for which not all of the rotated numbers were in its partition, which eliminated 11. 111, 1111, 11111, 111111 are all not prime. Then multiple the size of each partition by its number of digits, sum, add 1 for 11 gives the answer.

Problem 36 was a brute-force filter. Since leading zeros weren't allowed, even numbers could be eliminated, cutting the problem in half.

For problem 37, I built lists of n-digit primes that could be truncated from the left to n-1-digit primes, and lists of n-digit primes that could be truncated from the right to n-1-digit primes. The intersection of the left and right lists gives the 11 numbers.

From the example in problem 38, the first digit had to be a 9, so it was just a matter of searching 91..98, 912..987, 9123..9876, and 91234..98765.

For problem 39, p/3 < c < p-3, and c/2 < b < c-1. After that, I just did a brute-force search, which was kind of slow.

Problem 40 is just multiplying 7 single-digit numbers, and at least 2 of them are 1. It's straightforward to figure out those numbers, and, as it turns out, the product of those numbers is simple to mentally compute.

Monday, August 2, 2010

I wanted more immediate feedback about my Rock Band scores, so I modified the code I wrote that takes a picture of the television screen and does OCR (optical character recognition) to figure out what song was just played, and run text-to-speech with the name of the song and my high score. It works about half the time. But when it works, I get to know if I got a new high score without having to go over to the computer and check, which means I can be ready to play the next song.

Monday, July 26, 2010

Continuing to look at the Project Euler problems. The third 10 are a little more difficult than the previous 20.

My Haskell solution for problem 21 was too slow. So, in Java, I made a table of the sum of the divisors for the numbers from 1 to 9999, then added up the numbers that pointed at each other, as well as the numbers in the table that paired with numbers greater than 9999, and it was reasonably fast.

Problem 22 was straightforward.

I could not write a fast algorithm for problem 23 in Haskell. So, in Java, I made a table of whether the number is abundant for 1 to 28123. Then, each number from 1 to 28123 could be tested by testing for pairs of trues in the table where the indices added up to the number, which should be less than around 200 million tests, and short-circuiting after the first pair is found means the actual number of tests should be much lower than that.

Problem 24 doesn't require much computation. The first element of the nth permutation is the (n-1)/m!+1th element, where m+1 is the number of elements, and the rest of the nth permutation is the remainder of (n-1)/m!+1th permutation of the remaining elements.

Problem 25 was straightforward.

My solution to problem 26 was kind of slow, but not excessively slow. I calculated the cycle length by making a list of the remainders and searched the list of remainders for repeats with each new digit in the quotient. After that, it was a brute-force maximum search: snd $ maximum $ map (\ n -> (cycleLength n,n)) [1..1000].

My solution for problem 27 was kind of slow. b must be a prime under 1000 due to n=0, so there 168 possibilities for b. Given b and n, p = n2 + an + b, where p is prime, so a = (p - b)/n - n, so p - b has to be a multiple of n, so I made a list of primes satisfying that condition, which can be mapped to a list of values for a, which can be filtered for |a| < 1000. Then, I brute-force searched the 168 values of b for the maximum n and the corresponding a.

Problem 28 doesn't require much computation. The numbers on the corner of the nxn square, where n is odd, is a simple function of n, so the numbers on the diagonals can be added by recursively adding the diagonals of the (n-2)x(n-2) square.

For problem 29, any pair that has ai ≤ 100, where i > 1 and i divides b is not distinct and can be filtered out.

My solution for problem 30 was quite slow. Since 6×95 = 354294, 299999 is an upper bound. 25 + 5×95 = 295277, 15 + 5×95 = 295246, so I used 295246 as an upper bound for the brute force search. After working on problem 34, which is almost the same problem with a much bigger upper bound, I made the search faster by searching by 10s and only going through the 1s digits if the sum of the other digits is between the number + 9 - 95 and the number. This could be generalized to further prune the search by only considering the most significant digits, and only iterating through less significant digits if sum of the digits is within range of the number.

Monday, July 19, 2010

Continuing to look at the Project Euler problems. The second 10 are a little more difficult than the first 10.

Problem 11 is yet another search. I used transpose . zipWith drop [0..] to get a quarter of the diagonals, and combined it with various uses of reverse to get the rest of the diagonals. The result was on a diagonal.

My solution to problem 12 was horribly slow with hugs, so I solved it in Java. The algorithm I used should be O(N3/2), and the answer I got was between 50 million and 100 million. Perhaps my calculation of triangle numbers was O(N2) in hugs, instead of O(N), causing the calculation to be O(N5/2).

Problem 13 might have been more of a challenge when restricted to fixed size integer arithmetic. As it was, take 10 . show . sum.

My Haskell solution to problem 14 was horribly slow, so I solved it in Java by tabulating the results under a million and using the recursive formula whenever it went over a million.

My Haskell solution to problem 15 was horribly slow, so I solved it in Java by tabulating the results by building on the previously tabulated results for smaller grids, which should be O(N2).

Problem 16, like problem 13, was a simple one-liner that would have been much more difficult when restricted to fixed size integers.

Problem 17 is just a matter of counting correctly, without any nontrivial mathematical reasoning or computation.

Problem 18 (and problem 67) can be solved with pretty much a one-liner: foldr1 (\ row sums -> zipWith3 (\ i l r -> i + max l r) row sums (tail sums)).

Problem 19 is an easy brute-force iteration, with only 1200 months to consider. The 100 and 400 year rules can be ignored, given the range.

Problem 20 is a simple one-liner just like problem 16, where the challenge is mainly for those restricted to fixed-size integers.

Monday, July 12, 2010

When there was only busy-work at work, I started working through the Project Euler problems. For the most part, I used the hugs implementation of Haskell, since the problems ought to be solvable without a huge amount of computation. However, some of my solutions were way too slow, so I used Java to solve some of the problems. I haven't submitted any of my answers, so I don't have that verification that my answers are correct.

Problem 1 was a simple one-liner in Haskell

p1 = sum [3,6..999] + sum [5,10..999] - sum [15,30..999]


Problem 2 was a simple 2-liner in Haskell.

For problem 3, I started by computing a list of primes. After getting the answer, I realized that calculating the primes was superfluous. I could just divide out all factors, starting from 2, regardless if they were prime or not, and the remainder would be the largest prime factor.

I just brute forced problem 4, though I can halve the search space since multiplication is commutative.

Problem 5 was a simple one-liner. Haskell's Prelude provides gcd.

Problem 6 was another simple one-liner.

My solution for problem 7 was pretty slow. Adding a type declaration: primes :: [Int], made it over 3 times faster, though still slow, with hugs. The inferred type would have been primes :: [Integer], but since all of the relevant numbers fit into signed 32-bit integers, [Int] would still give the correct result. I sped up a number of slow solutions by adding Int declarations.

I brute forced problem 8. Since the last digit was a 0, I could just do product . take 5 and not worry about a spurious result, which I would have had to worry about if the number had ended with 09999.

Problem 9 was another brute-force search. Since 1000 > c > b > a > 0, I could limit the search to 1000 > c > 333 and c > b > c/2.

My solution to problem 10 in Haskell was just too slow using hugs, so I wrote basically the same algorithm in about 20 lines of Java, which was fast. The result had to be a long, since it was too big to for 32-bit integers. I manually stored the computed primes in an array in the Java solution. Perhaps hugs wasn't memoizing the list of primes that was being computed, causing the algorithm that was supposed to be O(N3/2) to be O(N2).

Monday, July 5, 2010

The service that I work on interfaces with things that other people work on. One problem that came up multiple times is that these other systems have hard-coded invalid assumptions that have worked so far, for various reasons. One of the reasons is that one of the early features that didn't fit those assumptions simply got reclassified as unsupported and has been removed from every release. Due to these problems, I'll have to account for these hard-coded assumptions in these other systems when adding new features in the future.

Monday, June 28, 2010

I got pulled into yet another demo, involving building yet another prototype. I managed to get something functional in under a week, in about 2000 lines of Java. The next two weeks involved rewriting lots of the code either due to my misunderstanding of what it was supposed to do, or changes in what it should do after having something available to play with. Since I was the only one working on the code, I felt free to rewrite big chunks of it. The demo ended up being about 3000 lines of Java and seems to have gone successfully.

Two things, which I've said before, apply to this experience.

  • Being able to throw away the first few iterations is valuable. The design of later iterations can incorporate issues that weren't forseen in the earlier iterations.

  • Static typing makes replacing large chunks of code manageable. The compiler caught numerous little oversights, whereas using a dynamically typed language would have been much more painful.


So, for prototyping and rapid iteration, I strongly prefer using statically typed languages.

For an established code base that has many people working on it, rewriting large chunks of code is less of an option.

  • It's harder to understand all of the issues in a large codebase.

  • Management conservatism discourages potentially disruptive changes.

Monday, June 21, 2010

After setting up my computer to take pictures of my television screen when finishing a Rock Band song, I can go back and see if I got any new high scores afterwards. However, the leaderboard interface is limited by the controllers and navigating hundreds of songs is clumsy, so I came up with a way to make the computer do most of that work. I can get my scores from rockbandscores.com in csv format, and use OCR (optical character recognition) software to get the name of the song out of the picture.

The scores and song names show up in two different ways. In the middle of a set, the song name appears at the bottom left of the screen. At the end of the set, the song name appears at the top of the screen, in uppercase, and in a different font. There are also lots of non-textual graphics on the screen. However, the OCR software doesn't have to be anywhere near perfect, since all it needs to do is to enable distinguishing between a limited set of a few hundred song titles. So I got ocrad, which is free, lightweight, and fast, though not very sophisticated or flexible.

I send the upper part of each image and the lower left of each image through OCR after processing the piece of the image into monochrome, selecting the whitest and brightest pixels. Originally, I selected for the brightest pixels, but I found that also selecting for the whitest pixels worked much better. There are still lots of non-text clutter left, which the OCR software interprets. So after gathering data on what results the OCR software got from various pictures, I made and refined a heuristic for matching the OCR results with the song titles.

Here are some initial results after getting it to work pretty well. First, a song in the middle of a setlist:


Processing the upper part and the lower left part causes the song title to stand out in the lower left.


The OCR software does a reasonable job on the lower left:

M1dn19htR1d9r. . .
\ a ' ;' '' .. , 1
_-__.


The OCR text is good enough that it matches one song in the scores.csv file, the correct one, and I did not get a new high score:

[Midnight Rider, The Allman Brothers Band, GUITAR:75218 6.18, DRUMS:127100 5.37]


After finishing the set:


Processing the upper part and lower left part of the image causes the song name and score to show up cleanly in the upper part:


The OCR software returns:

FRRNH11N'5 7DWER
225,225 _K1,v,v_

and the A in the picture does look like an R, and the K does look like an H, and all the other misread characters do look like the characters returned by the OCR software. The score was also correctly recognized, but that usually doesn't happen.

Matching the text with the scores.csv file, I got a new high score:

[Franklin's Tower, Grateful Dead, DRUMS:223275 5.98, GUITAR:155807 6.67]

Monday, June 14, 2010

One thing I like to do is play Rock Band. However, I often find that I don't remember what songs I just played in a playlist, and sometimes I want to check and see if I got a new high score. My first thought was to sniff the packets sent to the online leaderboards and pull the songs and the scores out of that data. Unfortunately, the packets are encrypted, so I gave up on that idea. But I had a new idea. I could point my computer's camera at the screen and record a video.

Making a video of the gameplay would eat up disk space and it would be a pain to go through a long video to see all the songs. But, going back to my first idea of sniffing packets, I could have some scheme of taking pictures and saving the ones around the time when packets were sent to the leaderboards. After finding out how to take pictures using the quicktime API and trying it out, I wrote some code that took a picture every 5 seconds, and interleaved the pictures with tcpdump output. I found that 32 byte packets were being sent every few seconds to Xbox Live, and 80 byte packets were being sent every once in a while, but larger packets were always sent after finishing a song. So then I wrote it to take a few pictures when seeing the larger packets. It worked really well. It takes some extraneous pictures due to other packets. Fortunately, I don't have lots of Xbox friends that would cause lots of notification packets.

Now I can play a dozen or so songs, and then go back and see how I did, instead of worrying about not remembering what I just played after playing only 3 or 4 songs.

import java.io.BufferedReader;
import java.io.File;
import java.io.FileOutputStream;
import java.io.InputStreamReader;
import java.io.PrintStream;
import java.util.Date;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.TimeUnit;
import quicktime.QTSession;
import quicktime.io.QTFile;
import quicktime.qd.Pict;
import quicktime.qd.QDConstants;
import quicktime.qd.QDGraphics;
import quicktime.qd.QDRect;
import quicktime.std.StdQTConstants4;
import quicktime.std.image.GraphicsExporter;
import quicktime.std.sg.SequenceGrabber;

public class Scorer {
private static ArrayBlockingQueue<Long> queue = new ArrayBlockingQueue<Long>(20);
private static boolean running = true;

private static void capturePhoto(String filename) throws Exception {
try {
QTSession.open();
QDRect qdRect = new QDRect(640, 480);
QDGraphics qdGraphics = new QDGraphics(qdRect);
SequenceGrabber sequenceGrabber = new SequenceGrabber();
sequenceGrabber.setGWorld(qdGraphics, null);
GraphicsExporter graphicsExporter = new GraphicsExporter(StdQTConstants4.kQTFileTypePNG);
graphicsExporter.setInputPicture(Pict.fromSequenceGrabber(sequenceGrabber, qdRect, 0, 0));
graphicsExporter.setOutputFile(new QTFile(filename));
graphicsExporter.doExport();
} finally {
QTSession.close();
}
}

private static void recorder(String dir) {
try {
PrintStream out = new PrintStream(new FileOutputStream(dir + "index.html"));
out.println("<html><body>");
int i = 0;
long t = 0L;
while (running) {
queue.take();
if (System.currentTimeMillis() - t < 30000L)
continue;
out.println("<br>Start:" + new Date() + "<br>");
for (int j = 0; j < 3; j++) {
capturePhoto(dir + i + ".png");
out.println("<img src=\"" + i + ".png\">");
i++;
TimeUnit.SECONDS.sleep(1);
}
out.println("<br>End:" + new Date() + "<br>");
queue.clear();
t = System.currentTimeMillis();
}
out.println("</body></html>");
out.flush();
out.close();
} catch (Exception e) {
e.printStackTrace();
}
}

private static void sniffer() {
try {
Process process = new ProcessBuilder("/usr/bin/sudo", "/usr/sbin/tcpdump", "-n", "-l", "port", "3074").start();
BufferedReader in = new BufferedReader(new InputStreamReader(process.getInputStream()));
while (running) {
String s = in.readLine();
if (s == null)
break;
if (s.endsWith("length 32") || s.endsWith("length 80"))
continue;
queue.offer(System.currentTimeMillis());
}
process.destroy();
} catch (Exception e) {
e.printStackTrace();
}
}

public static void main(String[] args) throws Exception {
new File("/tmp/score").mkdir();
new Thread() { public void run() { recorder("/tmp/score/"); } }.start();
new Thread() { public void run() { sniffer(); } }.start();
System.in.read();
running = false;
}
}

Monday, June 7, 2010

I played around a little with Clojure's Java interoperability at work. Since I made making a new module in the main product that I work on a matter of putting some classes and some xml in a jar file and putting that jar file and any jars it depends on in a directory, it shouldn't be a problem.

The Clojure code was all straightforward, using gen-class. How to actually get the class files generated was all in the documentation, but having the correct classpath was a big snag for me, especially the need to have ./classes in the classpath, which kept causing mysterious NoClassDefFoundErrors. After getting that figured out, I stuck it into ant:

<target name="compile">
<mkdir dir="${build.dir}"/>
<java classname="clojure.main">
<arg value="-e"/>
<arg value="(set! *compile-path* &#34;${build.dir}&#34;) (compile 'name.of.test.ModuleClass)"/>
<classpath>
<fileset dir="${lib.dir}">
<include name="*.jar"/>
</fileset>
<fileset dir="${clojure.home}">
<include name="*.jar"/>
</fileset>
<pathelement location="${src.dir}"/>
<pathelement location="${build.dir}"/>
</classpath>
</java>
</target>

Then, after getting the jar files built, I dropped the jar file and clojure.jar into the directory and got a giant stack trace, leading to

Caused by: java.io.FileNotFoundException: Could not locate clojure/core__init.class or clojure/core.clj on classpath:
at clojure.lang.RT.load(RT.java:402)
at clojure.lang.RT.load(RT.java:371)
at clojure.lang.RT.doInit(RT.java:406)
at clojure.lang.RT.(RT.java:292)
... 45 more

I hadn't the classloader set properly for the initial loading of the modules. For all the other modules, there wasn't much done in the class initializers, so this problem didn't come up. I had the classloader set for all other operations, though, so it was merely a matter of doing the same thing at initialization, and it worked.

I also noted that I handled the classloader by hand-creating a proxy class and implementing each method to wrap the classloader switch,

final ClassLoader handlerClassLoader = handler.getClass().getClassLoader();
return new Handler() {
public Object getObject(Object parameter) {
ClassLoader cl = Thread.currentThread().getContentClassLoader();
Thread.currentThread().setContextClassLoader(handlerClassLoader);
try {
return handler.getObject(parameter);
} finally {
Thread.currentThread().setContextClassLoader(cl);
}
}
};

which I could improve by using java.lang.reflect.Proxy, which I could use to wrap all the calls with one method. The code was about 100 lines shorter, and would no longer have to be changed if the Handler interface changed. I'll check in these changes, but not the test module in Clojure.

Monday, May 31, 2010

After playing with Clojure for a little while, one feature that I really like about it is doc and doc-strings, which work just like doc-strings in Emacs Lisp. I can use find-doc and ns-publics to find or explore the available symbols. I'm sure that an emacs mode or an IDE could make the interface as nice as the Emacs Lisp interface, but the functionality is there.

In Java, I often use javap to see what the method signatures of a class were, and have sometimes wished that the Javadoc were also available in the same way. The Java compiler could have put the Javadoc in attributes in the class files, which would then be available to tools like javap.

When using hugs or ghci for Haskell, I like using :browse and :info. The name and type almost always is enough to convey what a function does. It would be nice to have a short description of relevant information not captured by the name and type, but that's somewhat rare.

In any case, if I really needed to, documentation for the standard libraries for all three of these languages are available online. For Clojure and Haskell, the source code is available. For Java, the class files can be disassembled, though the occasional native method would remain opaque.

I find Clojure compelling in a way that I do not find Python compelling. They are both dynamically typed, which is a minus to me for both. I'm neither drawn to nor repelled by the syntax of either. However, I like feel of the immutable data structures of Clojure better than the data structures of Python. I'm also going to play around with integrating Clojure code with Java code. And while I haven't explored multimethods or any concurrency features, I can see learning new concepts, perhaps even learning when volatile would be useful when writing in Java, when looking into them.

Monday, May 24, 2010

In the product I work on at work, there are modules that accept items of various content types. Somebody made a dumb extension that made text content a special case, so now I'm getting silly NullPointerException bugs assigned to me because null is being passed in for the content. It would have been much cleaner to just use "text/plain" as the content type without any new special case code.

Monday, May 17, 2010

It seemed inappropriate that I hadn't implemented Parenthesis Hell in a Lisp, so I decided to learn a little about Clojure, a new Lisp dialect. I didn't use any of the concurrency features of Clojure, which is one of its main selling points. However, a Parenthesis Hell interpreter doesn't need any mutable state.

(defn ph-eval* [expr scope input]
(loop [scope* scope]
(cond
(empty? expr) input
(contains? scope* (first expr))
((scope* (first expr)) (rest expr) scope input scope*)
:else (recur (scope* 'outer)))))

(defn ph-valid? [val]
(cond
(not (seq? val)) false
(empty? val) true
(not (seq? (first val))) false
(empty? (first val)) (recur (rest val))
:else (recur (cons (first (first val)) (cons (rest (first val)) (rest val))))))

(defn ph-fn [body]
(fn [expr scope input def-scope]
(ph-eval* body def-scope expr)))

(defn ph-quote [expr scope input def-scope]
expr)

(defn ph-let [expr scope input def-scope]
(if (empty? expr)
()
(loop [bindings (first expr) scope* {'outer scope}]
(cond
(empty? bindings) (ph-eval* (rest expr) scope* input)
(empty? (first bindings)) (recur (rest bindings) scope*)
:else (recur (rest bindings)
(assoc scope*
(first (first bindings))
(ph-fn (rest (first bindings)))))))))

(defn ph-car [expr scope input def-scope]
(let [expr* (ph-eval* expr scope input)]
(if (empty? expr*) () (first expr*))))

(defn ph-cdr [expr scope input def-scope]
(let [expr* (ph-eval* expr scope input)]
(if (empty? expr*) () (rest expr*))))

(defn ph-cons [expr scope input def-scope]
(if (empty? expr)
()
(let [head (ph-eval* (first expr) scope input)
tail (ph-eval* (rest expr) scope input)]
(cons head tail))))

(defn ph-if [expr scope input def-scope]
(cond
(or (empty? expr) (empty? (rest expr))) ()
(not (empty? (ph-eval* (first expr) scope input)))
(ph-eval* (first (rest expr)) scope input)
:else
(ph-eval* (rest (rest expr)) scope input)))

(defn ph-eval [expr scope input def-scope]
(let [expr* (ph-eval* expr scope input)]
(if (not (ph-valid? expr))
(throw (IllegalArgumentException. "Not valid Parenthesis Hell")))
(ph-eval* expr* scope input)))

(defn ph-eval
"Evaluate a Parenthesis Hell expression with optional input."
([expr] (ph-eval expr ()))
([expr input]
(if (or (not (ph-valid? expr))
(not (seq? input)))
(throw (IllegalArgumentException. "Not valid Parenthesis Hell")))
(ph-eval* expr
{'() ph-quote
'(()) ph-let
'((())) ph-car
'(()()) ph-cdr
'((())()) ph-cons
'(()()()) ph-if
'(((()))) ph-eval}
input)))

Monday, May 10, 2010

Here is are a couple of programs in Parenthesis Hell that print themselves. I've inserted a bunch of line breaks, but they should be on one line.

The first one uses the optional concat in the initial scope. The second does not, and I haven't successfully tested it, since my interpreter runs out of memory when running it.

The following uses concat:

((())((((()()))()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()()()(
)(()(()()(()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()(
)()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()()()()(
()(()()(()()(()(()()()()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()
()()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()()()()((
)(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(
()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()((
)(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()
()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()()()()(()((
)()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()(()(
)(()(()()(()()(()(()()()()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()
()(()()(()(()()()()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()(()()
(()(()()(()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()
()()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()()()()((
)(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(
()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()((
)(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(
()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()()()()(()((
)()()()()(()(()()(()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()(
)(()(()()()()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()()()()(()(()(
)(()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()()()()((
)(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()(
()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()
(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()((
)()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(
()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()
()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(
()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()
()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()((
)()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()(
)(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()
()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()(
)(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()
()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()
(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()(
)()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()
(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()(
)()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(
()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()
()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(
()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()(
)()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()((
)(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(
()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()
(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()
()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(
()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()((
)()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(
()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()
()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()((
)()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()
()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()((
)()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()(
)(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()
()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()(
)(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()
()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()(
)(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()
()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()(
)(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()
()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()
(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()(
)()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(
()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()(
)(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(
()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()
()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()
(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()((
)()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()((
)(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()(
)(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(
()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()
()(()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()
(()(()()()()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()()()()(()(()()
()()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()()()()(()(()()()()()(
()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()(
()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()()()()(()(()()(()()(()
(()()()()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
))))))))))))))))((())(()()())()((()(()))(()()()(()(()()()())))(()(()))((())
((())))(()(()))(()()()(()(()()(()))))(())(()()))()()))(()(()))(()()()(()(()
()()()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()()()(
)(()(()()()()()(()(()()()()()(()(()()()()()(()(()()()()()(()(()()(()()(()((
)()()()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()()()()(()(()()(()))
))))))))))))))))))))))))))))))))))))))))(()(()))((())((()())))((()())))

The following does not use concat:

((())((((()()))()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()()()(
)(()(()()(()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()(
)()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()()()()(
()(()()()()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()
(()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()()()()(()(()()(()()((
)(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()()
()()(()(()()()()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()(()()(()
(()()(()()(()(()()()()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()((
)()(()(()()(()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()()()()(()(
()()()()()(()(()()(()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()(()(
)(()(()()()()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()
()()()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()()()(
)(()(()()(()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()()()()(()(()
()()()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()()()()(()(()()()()()
(()(()()()()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(
)()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()(()()(()
(()()(()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()()()()(()(()()()(
)()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()((
)()(()()(()(()()()()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()
()(()(()()(()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()()()()(()((
)()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()()()()(()(()()()()()
(()(()()()()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()()()()(()(()(
)(()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()(()()((
)(()()()()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()()()()(()(()()(
)()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()()()()(()(()()(()()(()
(()()(()()(()(()()()()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()
()()(()(()()(()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()()()()(()
(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()()()()(()(()()()()
()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()(
)()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()(()()(
()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()
(()()(()(()()(()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()()()()((
)(()()()()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()((
)()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(
()()()()()(()(()()(()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()
()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()()()()(()((
)()()()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()()()()(()(()()(()(
)(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()
()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()
(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()(
)()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()(()()((
)(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()()
()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()()()()(()
(()()(()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()()()()(()(()()()
()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()((
)()()()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()(()(
)(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()(
)()()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()()()()
(()(()()(()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()
()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(
()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()
(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()(()()(()
(()()(()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()()()()(()(()()()
()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()()()()(()((
)()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()(()()
(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()
(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()()()()((
)(()()()()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()()()()(()(()()(
()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()
(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()((
)()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(
()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()
()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(
()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()
()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()((
)()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()(
)(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()
()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()(
)(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()
()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()
(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()(
)()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()
(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()(
)()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(
()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()
()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(
()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()(
)()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()((
)(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(
()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()
(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()
()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(
()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()((
)()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(
()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()
()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()((
)()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()
()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()((
)()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()(
)(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()
()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()(
)(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()
()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()(
)(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()
()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()(
)(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()
()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()
(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()(
)()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(
()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()(
)(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(
()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()
()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(
()(()()(()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()()(
)()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()((
)(()()()()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()(
()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()
()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()
(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()
(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()((
)()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()(
)(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(
()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(
()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()
()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()()()()(()((
)()()()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()()()()(()(()()(()(
)(()(()()(()()(()(()()()()()(()(()()()()()(()(()()()()()(()(()()()()()(()((
)()()()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()()()()
(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(()(()()(()()(()(()(
)(()()(()(()()(()()(()(()()(()()(()(()()()()()(()(()()()()()(()(()()()()()(
()(()()(()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()(()()(()(()()((
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))((())(()()())(((())))((()()())(((()))((())))
(((())())((())((())())(((()))((())))(()()))())(()()())((()())((())))(((())(
))(())(())((())())((()())((())))(()()))(()()))(()()))((((())))(()()())()(((
))((())())(()()()(()(()()()())))(())((())())((((())))((())))(())((())())(()
()()(()(()()(()))))(((())))(()()))()()))(())((())())(()()()(()(()()()()()((
)(()()()()()(()(()()()()()(()(()()(()()(()(()()(()()(()(()()()()()(()(()()(
)()()(()(()()()()()(()(()()()()()(()(()()()()()(()(()()(()()(()(()()()()()(
()(()()(()()(()(()()(()()(()(()()(()()(()(()()()()()(()(()()(()))))))))))))
))))))))))))))))))))))))))))))(())((())())((((())))((()())))((()())))

Monday, May 3, 2010

The tasq programming language

tasq is a macro-expansion language with a single task queue and a small set of operations.
Syntax

A tasq program is a sequence of declarations or comments, optionally separated by whitespace.

A comment starts with a dot (.) and continues to the end of the line.

A declaration is an identifier followed by zero or more operations, and ending with a dot (.). Whitespace may separate identifiers and operations, but is otherwise ignored.

An operation is an identifier or one of +, -, ~, or ?.

An identifier is an uninterrupted sequence of non-whitespace characters not including +, -, ~, and ?.

A declaration of an identifier with zero following operations adds the identifier to the end of the task queue.

A declaration of an identifier followed by one or more operations defines the identifier's expansion as those operations.

It is illegal to use undefined identifiers. It is illegal to have more than one definition for an identifier.
Execution

Execution consists of dequeuing and executing the top item of the task queue repeatedly while the task queue is not empty.
Operations

+ Write 1 to the output.
- Write 0 to the output.
~ Discard the next item in the task queue.
? Read one bit from the input. If 1 is read, do nothing. If 0 is read, discard the next item in the task queue. If at EOF, discard the next two items in the task queue.
identifier Append the identifier's expansion to the end of the task queue.
Examples

Hello world:

w-+--+----++--+-+-++-++---++-++---++-++++--+------+++-
+++-++-++++-+++--+--++-++---++--+----+----+----+-+-.w.

cat:

bit? 1 0. .Read a bit
0 -bit. .Write 0, handle next bit
1 +~. .Write 1, discard the ensuing -
bit. .Initial task queue

Self-printing program (formatted as multiple lines, but should be a 1-liner):

d.0-z.1+o.z--+-------++----.o--+-------++---+.q p.p--+------+++---+--+-+++--
---+-+-.d 0 1 1 0 0 1 0 0 0 0 1 0 1 1 1 0 0 0 1 1 0 0 0 0 0 0 1 0 1 1 0 1 0
1 1 1 1 0 1 0 0 0 1 0 1 1 1 0 0 0 1 1 0 0 0 1 0 0 1 0 1 0 1 1 0 1 1 0 1 1 1
1 0 0 1 0 1 1 1 0 0 1 1 1 1 0 1 0 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1
0 1 1 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1
0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 1 1 0 0 1 0 1 0 1 1 0
0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 1 1
0 0 1 1 0 1 1 1 1 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 1 1 0 0 1 0 1
1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1
0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 1 1 0 0 1 0 1 0 1 1 0 0 1 0 1 1 0 1 0
0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 1 1 0 0 1 0 1 1 1 0 0 1 1 1 0 0 0
1 0 0 1 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 1 0 1 1 1 0 0 1 1 1 0 0 0 0 0 0 1 0 1
1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 1 1 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1
0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 1 1 0
0 1 0 1 0 1 1 0 0 1 0 1 0 1 1 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0
1 0 0 1 0 1 0 1 1 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 1 1 0 0 1 0 1
1 0 1 0 0 1 0 1 0 1 1 0 0 1 0 1 0 1 1 0 0 1 0 1 0 1 1 0 0 1 0 1 1 0 1 0 0 1
0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 1 0 1 0 0 1 0 1 0 1 1 0
0 1 0 1 1 0 1 0 0 1 0 1 0 1 1 0 0 1 0 1 1 0 1 0 0 1 0 1 1 1 0 0 1 1 0 0 1 0
0 q.

Interpreter


module Main(main) where

import Data.Map(Map,empty,insert,(!))
import Data.Bits(testBit)
import Data.Char(chr,ord)
import System.Environment(getArgs)
import Text.ParserCombinators.Parsec(CharParser,anyChar,char,getState,many,many1,manyTill,newline,noneOf,runParser,setState,space,spaces,(<|>),(<?>))

data Task = Out0 | Out1 | In | Discard | Task String

parse :: String -> String -> (String -> [Task],[Task])
parse file src =
either (error . show) id (runParser program (empty,[]) file src)

type Parser a = CharParser (Map String [Task],[Task]) a

program :: Parser (String -> [Task],[Task])
program = do
many (space <|> comment)
many statement
(map,queue) <- getState
return ((map !),reverse queue)

comment :: Parser Char
comment = do
char '.'
manyTill anyChar newline
return ' '

statement :: Parser ()
statement = do
id <- name
body <- many task
char '.'
many (space <|> comment)
(map,queue) <- getState
setState (if null body then (map,Task id:queue)
else (insert id body map,queue))

task :: Parser Task
task = do
t <- (char '-' >> return Out0)
<|> (char '+' >> return Out1)
<|> (char '?' >> return In)
<|> (char '~' >> return Discard)
<|> fmap Task name
spaces
return t

name :: Parser String
name = do
str <- many1 (noneOf "+-?~. \r\n\t\f\v")
spaces
return str

run :: (String -> [Task]) -> [Task] -> [Bool] -> [Bool]
run _ [] _ = []
run defs (Out0:tasks) input = False : run defs tasks input
run defs (Out1:tasks) input = True : run defs tasks input
run defs (In:[]) _ = []
run defs (In:_:[]) [] = []
run defs (In:_:_:tasks) [] = run defs tasks []
run defs (In:_:tasks) (False:input) = run defs tasks input
run defs (In:tasks) (True:input) = run defs tasks input
run defs (Discard:[]) _ = []
run defs (Discard:_:tasks) input = run defs tasks input
run defs (Task name:tasks) input = run defs (tasks ++ (defs name)) input

main :: IO ()
main = do
(file:_) <- getArgs
src <- readFile file
interact (fromBits . uncurry run (parse file src) . toBits)

toBits = concatMap (flip map [7,6..0] . testBit . ord)

fromBits (b7:b6:b5:b4:b3:b2:b1:b0:rest) =
chr ((if b7 then 128 else 0) + (if b6 then 64 else 0)
+ (if b5 then 32 else 0) + (if b4 then 16 else 0)
+ (if b3 then 8 else 0) + (if b2 then 4 else 0)
+ (if b1 then 2 else 0) + (if b0 then 1 else 0)) : fromBits rest
fromBits _ = []

Monday, April 26, 2010

Last year, I made a bunch of changes to automatically do a lot of the manual configuration that was involved in adding a new component and also managed to package everything to do with a component into a single jar file that gets its own ClassLoader so that different components can't interfere with each other. Two major things were making the SNMP configuration almost completely automatic, and automatically adding new database rows required for new components as needed. I did all this after two people involved in implementing the service left so I didn't have to worry too much about stepping on toes or having a bunch of bureaucratic meetings.

It has worked pretty well so far. The biggest snag was that some people ran a build that automatically added some rows to the database, then moved back to an older build that would refuse to initialize because database didn't match the configured components. This is no longer be an issue, as there aren't even builds that old in production.

However, there is another associated service that has a horrible tiered maven build that requires a bunch of configuration in two of the maven components as well as requiring new database rows be added for each new component. As long as somebody else is taking care of all of that, I don't care. But I've been called on to add the configuration when there are new components, and it's annoying to have to add almost 200 lines of boilerplate XML and request a minor database script update for each new component, and then go through the convoluted maven build process.

There are a few other associated services that, as yet, require manually adding stuff for each new component, but other people have been taking care of it. One of them is a hack onto a legacy infrastructure and is the source of most of the usage. Another two really ought to fetch everything associated with a component from the main service, which does have all the mechanisms required for doing so, but were implemented to have hard-coded copies of a few things for each component. But, for at least one of them, it's in the pipeline to do it right, after which it would no longer need any changes every time a new component is added.

The mentality that all new components need a database script update seems to be somewhat ingrained. Even today, someone asked me if they needed to run a database script, presumably for their development environment, as new components are being released to production in a few weeks. Their database probably already has had all the new rows for months, since the code for the new components have been in place for months.

Monday, April 19, 2010

The turn programming language

I decided to take a shot at making a two-dimensional programming language when noting that - and |, and / and \, and N and Z are 90-degree rotations of each other.

turn is a two-dimensional programming language that is invariant under 90-degree rotations and has any number of program counters.

A program counter (PC) has a location, direction, and turn direction.

A direction is up, right, down, or left.

A turn direction is left, straight, right, or u-turn.

Each cycle, each PC executes the operation at its location, then moves to its next location. If the next location is a wall and the turn direction is not straight, the direction is changed in the turn direction until the next location is not a wall. If the PC cannot turn to a location is not a wall, the PC dies. A PC also dies if it moves off the grid. If multiple PCs have the same location, direction, and turn direction, all but one of those PCs die. Execution ends when there are no remaining PCs.

There are 8 types of locations and their operations:
/: If the direction is horizontal, the turn direction rotates 90 degrees left. If the direction is vertical, the turn direction rotates 90 degrees right.
\: If the direction is horizontal, the turn direction rotates 90 degrees right. If the direction is vertical, the turn direction rotates 90 degrees left.
-: If the direction is horizontal, there is no effect. If the direction is vertical, the turn direction rotates 180 degrees.
|: If the direction is horizontal, the turn direction rotates 180 degrees. If the direction is vertical, there is no effect.
Z: If the direction is horizontal, read a bit from input. If the direction is vertical, write a bit to output.
N: If the direction is horizontal, write a bit to output. If the direction is vertical, read a bit from input.
+: Create a new PC at this location with direction in the turned direction and turn direction straight.
O: Read from this location or write to this location. If this location contains a bit, then read that bit, otherwise, write a bit. This location initially does not contain a bit.
space or dot (.): No-op.
^ > v <: No-op. Defines the starting location and direction of a PC. The initial turn direction is straight.
any other character: Wall. No-op.

When reading a bit, if the bit read is a 0, the turn direction is rotated 90 degrees to the left. If the bit read is a 1, the turn direction is rotated 90 degrees to the right. At EOF, the turn direction is rotates 180 degrees.

If multiple PCs read the input in a cycle, they all read the same bit. If multiple PCs read the same location in a cycle, they all read the same bit.

When writing a bit, if the turn direction is left, a 0 is written. If the turn direction is right, a 1 is written. Otherwise, nothing is written.

If multiple PCs are writing to the output in a cycle, if they all write the same value, then one bit with that value is written, otherwise nothing is written. If multiple PCs are writing to the same location in a cycle, if they all write the same value, then that value is written, otherwise nothing is written.

Examples

Hello world

>/N|N|NN|N|NNNN|NN|NN|N|N|N|N|NN|N|NN|NNN|NN|N|NN|NNN|NN|N|NNNN|NN|N|NNNNNN|NN#
#N|N|NNNN|N|NNNN|N|NNNN|N|NN|NN|NNN|NN|N|NN|NN|N|NN|NNN|N|NNNN|N|NN|N|NNN|N|Z
- #
N|Z
#

cat

_v_
+
ZNZ
( )
===

approximate touppercase

##|###|#||##---------#
#.......++.-.........#
##.###.#..#NO.......+#
##.......+|-.........#
##/###.#..##.........#
-.O......O...O......+#
##.###.#..##.........#
#.|N|#...............#
#.....|#.Z##.........#
AT##-#-#\.##.........#
PO.#+/+.+.....O......N
PU.#+.+.+......O.....N
RP.#+.+.+.......O....N
OP.#+.+.+........O...N
XE.#+.+.+.........O..N
IR.#+.+.+..........O.N
MC.#.#.#..##NNNNNNNN.#
AA\#+.+.+.>/++++++++.#
TS.\/\/...|APPROXIMATE
EE#######|#TOUPPERCASE

Interpreter


import java.io.BufferedReader;
import java.io.FileReader;
import java.io.InputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.util.ArrayList;
import java.util.Iterator;

public class Turn {
private int height;
private int width;
private ArrayList<ArrayList<Instruction>> grid = new ArrayList<ArrayList<Instruction>>();
private ArrayList<ArrayList<Boolean>> data = new ArrayList<ArrayList<Boolean>>();
private ArrayList<PC> pcs = new ArrayList<PC>();

public enum Instruction {
SLASH, BACKSLASH, DASH, VERT, N, Z, PLUS, O, NOOP, WALL,
}

public enum TurnDir {
STRAIGHT, RIGHT, UTURN, LEFT,
}

public class PC {
private int x;
private int y;
private int dx;
private int dy;
private TurnDir turnDir;

public PC(int x, int y, char dir) {
this.x = x;
this.y = y;
switch (dir) {
case '^': dx = 0; dy = -1; break;
case '>': dx = 1; dy = 0; break;
case 'v': dx = 0; dy = 1; break;
case '<': dx = -1; dy = 0; break;
default: assert false; break;
}
turnDir = TurnDir.STRAIGHT;
}

public PC(PC pc) {
this.x = pc.x;
this.y = pc.y;
this.dx = pc.dx;
this.dy = pc.dy;
this.turnDir = pc.turnDir;
doTurn();
this.turnDir = TurnDir.STRAIGHT;
}

public boolean inBounds() {
return x >= 0 && x < width && y >= 0 && y < height;
}

public void move() {
x += dx;
y += dy;
}

public Instruction rotate(Instruction insn) {
if (dx == 0)
return insn;
switch (insn) {
case SLASH: return Instruction.BACKSLASH;
case BACKSLASH: return Instruction.SLASH;
case DASH: return Instruction.VERT;
case VERT: return Instruction.DASH;
case N: return Instruction.Z;
case Z: return Instruction.N;
default: return insn;
}
}

public TurnDir getTurnDir() {
return turnDir;
}

public void setTurnDir() {
switch (rotate(grid.get(y).get(x))) {
case SLASH:
turnDir = TurnDir.values()[(turnDir.ordinal()+1)%4];
break;
case BACKSLASH:
turnDir = TurnDir.values()[(turnDir.ordinal()+3)%4];
break;
case DASH:
turnDir = TurnDir.values()[(turnDir.ordinal()+2)%4];
break;
}
}

public boolean facingWall() {
return x + dx >= 0 && x + dx < width
&& y + dy >= 0 && y + dy < height
&& grid.get(y + dy).get(x + dx) == Instruction.WALL;
}

public void doTurn() {
int t;
switch (turnDir) {
case RIGHT: t = dx; dx = -dy; dy = t; break;
case LEFT: t = dx; dx = dy; dy = -t; break;
case UTURN: dx = -dx; dy = -dy; break;
}
}

public boolean isInput() {
return rotate(grid.get(y).get(x)) == Instruction.N;
}

public void doInput(boolean eof, boolean bit) {
if (eof)
turnDir = TurnDir.values()[(turnDir.ordinal()+2)%4];
else if (bit)
turnDir = TurnDir.values()[(turnDir.ordinal()+1)%4];
else
turnDir = TurnDir.values()[(turnDir.ordinal()+3)%4];
}

public boolean isOutput() {
return (turnDir == TurnDir.LEFT || turnDir == TurnDir.RIGHT)
&& rotate(grid.get(y).get(x)) == Instruction.Z;
}

public boolean getOutput() {
return turnDir == TurnDir.RIGHT;
}

public boolean isSpawn() {
return grid.get(y).get(x) == Instruction.PLUS
&& turnDir != TurnDir.STRAIGHT;
}

public boolean isReadSignal() {
return grid.get(y).get(x) == Instruction.O
&& data.get(y).get(x) != null;
}

public void readSignal() {
doInput(false, data.get(y).get(x));
}

public void clearSignal() {
data.get(y).set(x, null);
}

public boolean isWriteSignal() {
return (turnDir == TurnDir.LEFT || turnDir == TurnDir.RIGHT)
&& grid.get(y).get(x) == Instruction.O
&& data.get(y).get(x) == null;
}

public void setSignal() {
data.get(y).set(x, getOutput());
}

public boolean equals(PC pc) {
return pc.x == x && pc.y == y && pc.dx == dx && pc.dy == dy
&& pc.turnDir == turnDir;
}

public boolean sameLocation(PC pc) {
return pc.x == x && pc.y == y;
}
}

public Turn(BufferedReader in) throws IOException {
String line;
while ((line = in.readLine()) != null)
parseLine(line);
height = grid.size();
width = 0;
for (ArrayList<Instruction> row : grid)
width = Math.max(width, row.size());
for (ArrayList<Instruction> row : grid)
while (row.size() < width)
row.add(Instruction.NOOP);
while (data.size() < height)
data.add(new ArrayList<Boolean>());
for (ArrayList<Boolean> row : data)
while (row.size() < width)
row.add(null);
}

private void parseLine(String line) {
ArrayList<Instruction> row = new ArrayList<Instruction>();
for (int i = 0; i < line.length(); i++)
switch (line.charAt(i)) {
case '^': case '>': case 'v': case '<':
pcs.add(new PC(i, grid.size(), line.charAt(i)));
/*FALLTHROUGH*/
case ' ': case '.':
row.add(Instruction.NOOP);
break;
case '/': row.add(Instruction.SLASH); break;
case '\\': row.add(Instruction.BACKSLASH); break;
case '-': row.add(Instruction.DASH); break;
case '|': row.add(Instruction.VERT); break;
case 'N': row.add(Instruction.N); break;
case 'Z': row.add(Instruction.Z); break;
case '+': row.add(Instruction.PLUS); break;
case 'O': row.add(Instruction.O); break;
case 'X': row.add(Instruction.WALL); break;
default: row.add(Instruction.WALL); break;
}
grid.add(row);
}

public void run(BitInputStream in, BitOutputStream out) throws IOException {
ArrayList<PC> list = new ArrayList<PC>();
ArrayList<PC> list2 = new ArrayList<PC>();
for (;;) {
for (PC pc : pcs)
if (pc.isOutput())
list.add(pc);
boolean doOutput = false;
boolean output = false;
for (PC pc : list)
if (doOutput) {
if (output != pc.getOutput()) {
doOutput = false;
break;
}
} else {
doOutput = true;
output = pc.getOutput();
}
if (doOutput)
out.write(output);
list.clear();

for (PC pc : pcs) {
pc.setTurnDir();
if (pc.isInput())
list.add(pc);
}

if (list.size() > 0) {
boolean eof = in.eof();
boolean bit = in.read();
for (PC pc : list)
pc.doInput(eof, bit);
list.clear();
}

for (PC pc : pcs) {
if (pc.isReadSignal()) {
pc.readSignal();
list2.add(pc);
continue;
} else if (!pc.isWriteSignal() || list.contains(pc)) {
continue;
}
doOutput = true;
for (PC pc2 : pcs)
if (pc != pc2 && pc.sameLocation(pc2)) {
list.add(pc2);
if (pc.getOutput() != pc2.getOutput())
doOutput = false;
}
if (doOutput)
pc.setSignal();
}
list.clear();
for (PC pc : list2)
pc.clearSignal();
list2.clear();

for (PC pc : pcs)
if (pc.isSpawn())
list.add(new PC(pc));
pcs.addAll(list);
list.clear();

loop:
for (Iterator<PC> i = pcs.iterator(); i.hasNext(); ) {
PC pc = i.next();
for (PC pc2 : pcs)
if (pc2 != pc && pc2.equals(pc)) {
i.remove();
continue loop;
}
if (pc.getTurnDir() != TurnDir.STRAIGHT) {
for (int n = 0; n < 4 && pc.facingWall(); n++)
pc.doTurn();
if (pc.facingWall())
i.remove();
}
}

for (Iterator<PC> i = pcs.iterator(); i.hasNext(); ) {
PC pc = i.next();
pc.move();
if (!pc.inBounds())
i.remove();
}

if (pcs.isEmpty())
break;
}
out.flush();
}

public static class BitInputStream {
private InputStream in;
private int bit;
private int octet;

public BitInputStream(InputStream in) {
this.in = in;
this.bit = 0;
this.octet = 0;
}

public boolean eof() throws IOException {
if (octet >= 0 && bit == 0) {
bit = 128;
octet = in.read();
}
return octet < 0;
}

public boolean read() throws IOException {
if (octet >= 0 && bit == 0) {
bit = 128;
octet = in.read();
}
try {
return (octet & bit) != 0;
} finally {
bit >>>= 1;
}
}

public void close() throws IOException {
in.close();
}
}

public static class BitOutputStream {
private OutputStream out;
private int bit;
private int octet;

public BitOutputStream(OutputStream out) {
this.out = out;
this.bit = 128;
this.octet = 0;
}

public void write(boolean b) throws IOException {
if (b)
octet |= bit;
bit >>>= 1;
if (bit == 0) {
out.write(octet);
bit = 128;
octet = 0;
}
}

public void close() throws IOException {
out.close();
}

public void flush() throws IOException {
out.flush();
}
}

public static class ASCIIBitOutputStream extends BitOutputStream {
private OutputStream out;

public ASCIIBitOutputStream(OutputStream out) {
super(out);
this.out = out;
}

public void write(boolean b) throws IOException {
out.write(b ? '1' : '0');
}
}

public static void main(String[] args) throws Exception {
String src;
BitOutputStream out;
if ("-b".equals(args[0])) {
src = args[1];
out = new ASCIIBitOutputStream(System.out);
} else {
src = args[0];
out = new BitOutputStream(System.out);
}
new Turn(new BufferedReader(new FileReader(src))).run(new BitInputStream(System.in), out);
}
}