Monday, January 31, 2011

I've integrated purchasing with Facebook Credits into a Facebook application. In the end, the API was good, but the documentation was terrible, making it hard to get started. To start with, I needed to handle a callback from Facebook, and the documentation said
There are two callbacks Facebook makes on the application back end. The application needs to verify the fb_sig parameter to make sure that the request is coming from facebook.

I then had to guess and use trial-and-error to get it to work. What the documentation should have said, but didn't say was
  • The callback is an HTTP POST.
  • The content-type of the posted content is application/x-www-form-urlencoded.
  • The two callbacks are indicated by the method parameter.
  • The fb_sig parameter only needs to be verified if the application does not have the OAuth 2.0 for Canvas setting enabled. If that setting is enabled, the fb_sig parameter is not sent, and all the parameters are in the signed_request, which includes a signature that needs to be verified.
  • The order_details parameter is a string containing the original JSON, which means it needs to be double parsed.
  • They provide an example for the response to the payments_get_items callback, but not for the payments_status_update callback. Following the given example for the payments_status_update response results in an unhelpful error message to the user, with no feedback pointing to the problem. As the documentation was unhelpful, and the Facebook developer forums had a few posts from someone facing the same problem with no response, I resorted to trial-and-error. (I'm not creating yet another account and password just to post to that forum.) The problem was that the content field in the payments_get_items response is supposed to be an array, but it is supposed to be a single item in the payments_status_update response.

Monday, January 24, 2011

I've said that I've found the Python programming language uncompelling. As a language, my impression is that it's a better language than perl, but I still use perl from time to time, but I don't use Python. The main way I use perl is from the command line to run one-off scripts. The one thing I knew about Python from the early 1990s when I first heard about it, until 2009, when I decided to try using it, was that indentation was syntactically significant, which means you can't write one-liners (even though the one line could contain 500 or more characters) like you can with braces and semicolons. perl -e 'while (<>) { ... }' is very convenient when trying to do something that would be too complicated when using pure shell constructs.

Monday, January 17, 2011

I found out that in Tomcat 6, HttpServletRequest.getRequestURL() omits the ";jsessionid=[sessionid]", if present, while Tomcat 7 returns it. In either case, HttpServletResponse.encodeRedirectURL() adds it in or replaces it with the actual session id when the client has cookies disabled. However, what I needed was to have the ";jsessionid=[requested sessionid]", and not the actual session id, because I needed to reconstruct the URL that was actually sent for the redirect_uri parameter when obtaining an OAuth 2.0 access token.

What had happened was that when multiple hosts were added to the load balancer, clients with cookies disabled would get authentication failures. Since this was a new service in private beta, the immediate fix was to have a single host in the load balancer. I then would add a second host to the load balancer, do a quick test, then remove the second host from the load balancer, and then look at the logs to see what was going on. I had originally constructed the redirect_uri parameter using HttpServletResponse.encodeRedirectURL() on the results of HttpServletRequest.getRequestURL() and HttpServletRequest.getQueryString(). It worked fine with clients that had cookies enabled, because the session id wouldn't be in the URL, and the load balancer used cookies for host stickiness. However, for clients with disabled cookies, the load balancer would bounce the client between hosts for subsequent requests, and since sessions weren't really being shared between hosts, so when a client request gets sent to a different host, it gets a new session. (I did have a scheme to recover a very small set of essential session data through memcached, but I didn't think that saving redirect_uri in it was necessary, since it ought to be available when the client is retrieving that exact URL. I also should have a scheme to make sure the session ids from one host never collide with session ids from another host by prepending the hostname to the session id.)

After I figured out what was happening, I removed the HttpServletResponse.encodeRedirectURL() call in the construction of the redirect_uri parameter. It seemed fine to me, because I was using Tomcat 7. However, once it was running on other systems, there immediate failures for clients with disabled cookies. From the logs, I could tell that HttpServletRequest.getRequestURL() was omitting the ";jsessionid=[session id]", so I theorized that it was because of differences between Tomcat 6 and Tomcat 7, and quickly confirmed it by running Tomcat 6 myself.

I finally settled on an ugly hack, where if HttpServletRequest.isRequestedSessionIdFromURL() and if ":jsessionid=" were not in HttpServletRequest.getRequestURL(), I would add ";jsessionid=" + HttpServletRequest.getRequestedSessionId(), which would then work on Tomcat 6 and Tomcat 7.

Monday, January 10, 2011

When doing some shell scripting for build and deploy processes, I ran into something I found strange and confusing. I had a loop

while read host; do ssh -i id $host command; done < hosts

However, it only executed the remote command on the first host, which was odd. I threw in some echos to see if something strange was happening, but it only confirmed that the loop only went through one iteration. Then I changed ssh to echo, and the loop iterated through all the hosts. Finally, I figured that ssh was slurping up stdin for some reason, causing the loop to end, and fixed the problem with

while read host; do ssh -n -i id $host command; done < hosts

Monday, January 3, 2011

I created a simple web application and put it up on Amazon Web Services for about a week. The cost for a single instance and a single load balancer was around $21, including a few cents for storage and traffic. It would be about $82 a month. So I ported it to Google App Engine, where a low-traffic web application could be hosted for free. It took about a day. I replaced the storage layer that used Amazon S3 with one that used JDO. I replaced the memcached layer using the net.spy.memcached client with one using the JCache (JSR107) API. Those were straightforward and most of the rest of the code didn't need any changes. I also added the configuration files appengine-web.xml and jdoconfig.xml.

I ran into a few snags with the appengine servlet container, some of which might have been due to strange interactions with Google Guice, but that's just speculation without any real investigation.

The first one is probably a bug in the appengine (or jetty) implementation of HttpServletResponse.encodeRedirectURL(). It erroneously added ;jsessionid=sessionid to external URLs. I worked around that by not calling encodeRedirectURL() for external URLs, even though the javadoc says "All URLs sent to the HttpServletResponse.sendRedirect method should be run through this method."

The next one seemed like some weird difference in implementations. I used guice-servlet to configure servlet filters on /index.jsp, but they weren't being invoked for /, while the filters were being invoked under Tomcat and Glassfish. I worked around that by changing filter("/index.jsp").through(MyFilter.class) to filter("/","/index.jsp").through(MyFilter.class).

The last one was the weirdest, and has to be some kind of bug in either appengine, jetty, or guice. Once I got the filters passing the request for / through, instead of serving up /index.jsp, the service returned a redirect to //, which turned into a redirect to ///, etc, until the browser stopped due to too many redirects. I worked around that by kludging in a special servlet for /:

@Singleton
public static class RedirectToIndexJSP extends javax.servlet.http.HttpServlet {
private static final long serialVersionUID = 0L;
@Override
protected void doGet(javax.servlet.http.HttpServletRequest request, javax.servlet.http.HttpServletResponse response) throws javax.servlet.ServletException, java.io.IOException {
request.getRequestDispatcher("/index.jsp").forward(request, response);
}
}

After that, it all worked. Compared to Amazon, the service on Google App Engine is a lot slower for the first request, as this is a very low-traffic application, so it's pretty much never running, and a new virtual machine starts up when a request does come in, which seems to take around 10 seconds. Subsequent requests are reasonable, though still slower than Amazon.