The Parenthesis Hell programming language

The Parenthesis Hell programming language is an obfuscated, Lisp-like programming language. The only values are a cons pair or nil, which can be represented by using a subset of S-expressions. Since nil can be written as (), the dotted pair notation is superfluous, and everything can be written in list notation. Thus, in Parenthesis Hell, everything must be written in list notation, so everything Parenthesis Hell consists of matched pairs of open and close parentheses. All other characters are ignored.

I will use the dotted-pair notation for clarity. In particular, it highlights certain ways Parenthesis Hell differs from Lisp. However, it must be translated to list notation to be legal Parenthesis Hell.

A Parenthesis Hell program consists of a single expression. Its argument is the program's input. The program's output is the value of the expression.

An expression is either nil, written as (), or a cons pair.

nil, or (), evaluates to the current argument.

A cons pair, (head . tail), is evaluated by looking up head in the current scope, with tail as its argument.

The initial scope defines 7 functions.

()	quote	(() . expr)
(())	letrec	((()) (definition-list) . expr)
(()())	cdr	((()()) . expr)
(()()())	if	((()()()) cond expr1 . expr2)
((()))	car	(((())) . expr)
((())())	cons	((()()) expr1 . expr2)
(((())))	eval	((((()))) . expr)

quote, (() . expr), evaluates to the literal expr.

letrec, ((()) (definition-list) . expr), creates a new scope with given definitions, which may shadow definitions in the enclosing scopes, including those in the initial scope, and evalutes to the evaluation of expr in the new scope.

cdr, ((()()) . expr), evaluates to the tail of the evaluation of expr in the current scope. If expr evaluates to nil, the value is nil.

if, ((()()()) cond expr1 . expr2), evaluates to the evaluation of expr1 in the current scope if the evaluation of cond in the current scope is not nil, otherwise it evaluates to the evaluation of expr2 in the current scope. If (cond expr1 . expr2) is nil or if (expr1 . expr2) is nil, the value is nil.

car, (((())) . expr), evaluates to the head of the evaluation of expr in the current scope. If expr evaluates to nil, the value is nil.

cons, (((())()) expr1 . expr2), evaluates to the cons of the evaluation of expr1 and the evaluation of expr2 in the current scope.
If (expr1 . expr2) is nil, the value is nil.

eval, ((((()))) . expr), evaluates to the evaluation of the evaluation of expr in the current scope. Obligatory in languages where code is data.

Creating new scopes

In a new letrec scope, ((()) (definition-list) . expr), definition-list is a list of (name . body), where name is defined
within the scope, which includes the bodies in the definition-list. When name is invoked as (name . expr), the evaluation of expr in the current scope becomes the argument, referred to by (), in scope of name's body. The value of (name . expr) is value of body evaluated in the scope of name's body. nils in the definition-list are ignored.

Extensions

The initial scope can optionally define additional functions.

concat

concat, ((()(())) expr1 . expr2), takes the evaluation of expr1 and the evaluation expr2 in the current scope as output, and returns a value that would output the bits of expr1, followed by the bits of expr2. An approximate implementation in Parenthesis Hell, which does not evaluate expr1 and expr2 separately, is

((())(((())(()()())(((())))((()()())(((()))((())))(((())())(((
))((())())(((()))((())))(()()))())(()()())((()())((())))(((())
())(())(())((())())((()())((())))(()()))(()()))(()())))(()))

(ph-concat . (((())()) expr1 . expr2)), or (ph-concat . (cons expr1 . expr2)), is equivalent to (concat expr1 . expr2). The Parenthesis Hell implementation can use huge amounts of memory, having implementing concat in the interpreter enables some code that would otherwise cause out of memory errors.

Example code

cat:

()

Hello world:

(()()(()()(()()()()((()()(()(()((()((()()()((()((()()()((()((((()()(()(
)()()()()(((()(((()((()((((()(((()()(()()((()((()()()((()()(()()()()(()
()()()(()()()()(()(())))))))))))))))))))))))))))))))))))))))))))))))))

quine:

((())(((())()((())())(()())((())())(((())())(((())())(()())((())())(())(()))()
)(())))((())())(()())((())())(((())())(((())())(()())((())())(())(()))())(()))

An interpreter, written in Haskell

I wrote it in multiple modules, but crammed everything into a single module for this post.

module Main(main) where

import Data.Bits(bit,testBit)
import Data.Char(chr,ord)
import Prelude hiding (lookup,head,tail)
import System.Environment(getArgs)

data Value = Nil | Cons Value Value

instance Show Value where
    show a = '(' : shows a ")" where
        shows Nil s = s
        shows (Cons a b) s = '(' : shows a (')' : shows b s)

isNil :: Value -> Bool
isNil Nil = True
isNil _ = False

car :: Value -> Value
car (Cons a _) = a
car Nil = Nil

cdr :: Value -> Value
cdr (Cons _ b) = b
cdr Nil = Nil

instance Read Value where
    readsPrec p s@('(':_) = [readNext s]
    readsPrec p (_:s) = readsPrec p s
    readsPrec p [] = []

readNext :: String -> (Value,String)
readNext ('(':s) = readNext' s
  where
    readNext' ('(':s) =
        let (car,s') = readNext' s
            (cdr,s'') = readNext' s'
        in  (Cons car cdr,s'')
    readNext' (')':s) = (Nil,s)
    readNext' (_:s) = readNext' s
    readNext' [] = error "unmatched ("

readNext (')':_) = error "unmatched )"
readNext (_:s) = readNext s
readNext [] = error "unmatched ("

strToValue :: String -> Value
strToValue [] = Cons Nil Nil
strToValue (c:cs) = bitsToValue [7,6..0] c
  where
    bitsToValue [] _ = strToValue cs
    bitsToValue (b:bs) byte =
        (if testBit (ord c) b then Cons else flip Cons)
            (bitsToValue bs byte) Nil

valueToStr :: Value -> String
valueToStr value = valueToBits [7,6..0] 0 value
  where
    valueToBits [] byte rest = chr byte : valueToStr rest
    valueToBits _ _ Nil = []
    valueToBits (_:bs) byte (Cons Nil rest) = valueToBits bs byte rest
    valueToBits (b:bs) byte (Cons rest _) = valueToBits bs (byte + bit b) rest

data ValueTrie a = ValueTrie (ValueTrie a) (ValueTrie a) (Maybe a) | Empty

head Empty = Empty
head (ValueTrie hd _ _) = hd

tail Empty = Empty
tail (ValueTrie _ tl _) = tl

value Empty = Nothing
value (ValueTrie _ _ val) = val

empty :: ValueTrie a
empty = Empty

insert ::  ValueTrie a -> Value -> a -> ValueTrie a
insert trie key val =
    let insertValue trie = ValueTrie (head trie) (tail trie) (Just val)
    in  insert' trie key insertValue

insert' :: ValueTrie a -> Value -> (ValueTrie a -> ValueTrie a) -> ValueTrie a
insert' trie Nil insertValue = insertValue trie
insert' trie (Cons hd tl) insertValue =
    let insertTail trie' =
            ValueTrie (head trie')
                      (insert' (tail trie') tl insertValue)
                      (value trie')
    in  ValueTrie (insert' (head trie) hd insertTail) (tail trie) (value trie)

lookup :: ValueTrie a -> Value -> Maybe a
lookup = lookup' value

lookup' :: (ValueTrie a -> b) -> ValueTrie a -> Value -> b
lookup' value Empty _ = value Empty
lookup' value trie Nil = value trie
lookup' value trie (Cons hd tl) =
    case lookup' id (head trie) hd of
        Empty -> value Empty
        trie' -> lookup' value (tail trie') tl

data Scope a b = Root | Nested (Scope a b) (a -> Maybe b)

root :: (a -> Maybe b) -> Scope a b
root = Nested Root

nested :: Scope a b -> (a -> Maybe b) -> Scope a b
nested = Nested

get :: Show a => Scope a b -> a -> b
get scope@(Nested outer getValue) name =
    case getValue name of
        Just value -> value
        Nothing    -> get outer name
get Root name = error (show name)

data Op = Op (Value -> Scope Value Op -> Value -> Value)
op (Op op) = op

rootScope :: Scope Value Op
rootScope = root getBinding where
    getBinding Nil = Just (Op quoteOp)
    getBinding (Cons Nil Nil) = Just (Op letOp)
    getBinding (Cons Nil (Cons Nil Nil)) = Just (Op cdrOp)
    getBinding (Cons Nil (Cons Nil (Cons Nil Nil))) = Just (Op ifOp)
    getBinding (Cons Nil (Cons (Cons Nil Nil) Nil)) = Just (Op concatOp)
    getBinding (Cons (Cons Nil Nil) Nil) = Just (Op carOp)
    getBinding (Cons (Cons Nil Nil) (Cons Nil Nil)) = Just (Op consOp)
    getBinding (Cons (Cons (Cons Nil Nil) Nil) Nil) = Just (Op evalOp)
    getBinding _ = Nothing

quoteOp input scope arg = arg

letOp input scope Nil = Nil
letOp input scope (Cons bindings letBody) =
    let bindingList Nil = empty
        bindingList (Cons Nil rest) = bindingList rest
 bindingList (Cons (Cons name body) rest) =
            let definedOp input scope arg =
                    eval (eval input scope arg) letScope body
            in  insert (bindingList rest) name (Op definedOp)
        letScope = nested scope (lookup (bindingList bindings))
    in  eval input letScope letBody

cdrOp input scope arg = cdr (eval input scope arg)

ifOp input scope Nil = Nil
ifOp input scope (Cons _ Nil) = Nil
ifOp input scope (Cons cond body)
    | isNil (eval input scope cond) = eval input scope (cdr body)
    | otherwise                     = eval input scope (car body)

carOp input scope arg = car (eval input scope arg)

consOp input scope Nil = Nil
consOp input scope (Cons head tail) =
    Cons (eval input scope head) (eval input scope tail)

evalOp input scope arg = eval input scope (eval input scope arg)

concatOp input scope Nil = Nil
concatOp input scope (Cons head tail) =
    let concat Nil rest = rest
 concat (Cons Nil Nil) rest = rest
        concat (Cons Nil tl) rest = Cons Nil (concat tl rest)
        concat (Cons hd tl) rest = Cons (concat hd rest) tl
    in  concat (eval input scope head) (eval input scope tail)

eval input scope Nil = input
eval input scope expr = (op (get scope (car expr))) input scope (cdr expr)

main :: IO ()
main = do
    args <- getArgs
    (toStr,code,arg) <- processArgs valueToStr args
    putStr $ toStr $ eval arg rootScope code

processArgs :: (Value -> String) -> [String] -> IO (Value -> String,Value,Value)
processArgs toStr [] = do
    src <- getContents
    return (toStr, read src, Nil)
processArgs _ ("-v":args) = processArgs show args
processArgs toStr (file:_) = do
    src <- readFile file
    input <- getContents
    return (toStr, read src, strToValue input)

It had been over 10 years since I had last written anything in Haskell, so I decided to try using it to see if I still could. I returned to the simple 01_ programming language that I invented earlier this year, and wrote an interpreter, and it took less than a day. I didn't worry about making it fast, but the interpreter compiled with ghc was about twice as fast as the one I wrote in C

I had written an interpreter in Java, and the source was about 3 times longer than the Haskell version. Since Haskell has laziness built-in, I didn't have to code in the laziness. The source for the interpreter in C that I wrote was about twice the size of the Java version and about 6 times the Haskell version. A lot of the C code was junk for debugging the memory management, since I couldn't rely on a garbage collector.

Haskell	216 lines
Java	782 lines
C	1527 lines

I first wrote the Haskell 01_ interpreter in multiple modules, and it would be more maintainable that way. However, I'll present here all smashed into a single module, so that it can be stuck into a single file and run.

module Main(main) where

import Data.Bits(bit,testBit)
import Data.Char(chr,ord)
import Data.List(splitAt,elemIndex,(!!))
import Data.Map(Map,empty,findWithDefault,insert,member,(!))
import qualified Data.Map(map)
import System.Environment(getArgs)

data Token = Symbol String | Zero | One | Equal | Dot | Nil

tokenize :: String -> [Token]
tokenize ('=':'=':cs) = tokenize (dropComment cs)
tokenize ('0':cs) = Zero : tokenize cs
tokenize ('1':cs) = One : tokenize cs
tokenize ('=':cs) = Equal : tokenize cs
tokenize ('.':cs) = Dot : tokenize cs
tokenize ('_':cs) = Nil : tokenize cs
tokenize (' ':cs) = tokenize cs
tokenize ('\t':cs) = tokenize cs
tokenize ('\r':cs) = tokenize cs
tokenize ('\n':cs) = tokenize cs
tokenize [] = []
tokenize cs = tokenizeSymbol cs ""

tokenizeSymbol s@(c:cs) symbol =
    case c of
        '=' -> Symbol (reverse symbol) : tokenize s
        '0' -> Symbol (reverse symbol) : tokenize s
        '1' -> Symbol (reverse symbol) : tokenize s
        '.' -> Symbol (reverse symbol) : tokenize s
        '_' -> Symbol (reverse symbol) : tokenize s
        ' ' -> Symbol (reverse symbol) : tokenize cs
        '\t' -> Symbol (reverse symbol) : tokenize cs
        '\r' -> Symbol (reverse symbol) : tokenize cs
        '\n' -> Symbol (reverse symbol) : tokenize cs
        _ -> tokenizeSymbol cs (c:symbol)
tokenizeSymbol [] symbol = [Symbol (reverse symbol)]

dropComment ('\n':cs) = cs
dropComment (_:cs) = dropComment cs
dropComment [] = []

type Bits = [Bool]
data Pattern = Literal Bits | Binding Bits String | Wildcard Bits
data Expr = LiteralBits Bits | Concat Expr Expr | Bound Int | Call String [Expr]
data Definition = Def [Pattern] Expr

data Signature = Sig String [Pattern] [Token]

parseSigs :: [Token] -> [Signature]
parseSigs (Symbol s:tokens) = parseSig s [] tokens []
parseSigs [] = []
parseSigs _ = error "name expected to start definition"

parseSig name patterns (Equal:tokens) [] =
    collectBody name (reverse patterns) tokens []
parseSig name patterns (Equal:tokens) bits =
    collectBody name (reverse (Literal (reverse bits) : patterns)) tokens []
parseSig name patterns (Zero:tokens) bits =
    parseSig name patterns tokens (False:bits)
parseSig name patterns (One:tokens) bits =
    parseSig name patterns tokens (True:bits)
parseSig name patterns (Nil:tokens) bits =
    parseSig name (Literal (reverse bits) : patterns) tokens []
parseSig name patterns (Symbol binding:tokens) bits =
    parseSig name (Binding (reverse bits) binding : patterns) tokens []
parseSig name patterns (Dot:tokens) bits =
    parseSig name (Wildcard (reverse bits) : patterns) tokens []

collectBody name patterns (Dot:tokens) body =
    Sig name patterns (reverse body) : parseSigs tokens
collectBody name patterns (token:tokens) body =
    collectBody name patterns tokens (token:body)
collectBody name patterns [] body = error ("unexpected end of body of "++name)

groupDefs [] defs = Data.Map.map reverse defs
groupDefs (sig@(Sig name patterns _):sigs) defs =
    case findWithDefault [] name defs of
        [] -> groupDefs sigs (insert name [sig] defs)
        siglist@(Sig _ otherPatterns _:_) ->
            if length patterns /= length otherPatterns then
                error ("arity mismatch:"++name)
            else
                groupDefs sigs (insert name (sig:siglist) defs)

parseDefs defs = Data.Map.map (map (parseDef arities)) defs
    where
        arities = Data.Map.map (\ (Sig _ patterns _:_) -> length patterns) defs

parseDef arities (Sig _ patterns body) =
    case body of
        [] -> Def patterns (LiteralBits [])
        _ -> Def patterns
            (foldr1 (\a b -> Concat a b) (parseExprs arities bindings body))
    where
        bindings = map (\ (Binding _ name) -> name)
            (filter (\x -> case x of Binding _ _ -> True; _ -> False) patterns)

parseExprs arities bindings [] = []
parseExprs arities bindings tokens@(Zero:_) =
    parseLiteral arities bindings tokens []
parseExprs arities bindings tokens@(One:_) =
    parseLiteral arities bindings tokens []
parseExprs arities bindings tokens@(Nil:_) =
    parseLiteral arities bindings tokens []
parseExprs arities bindings (Symbol name:tokens) =
    case elemIndex name bindings of
        Just i -> Bound i : parseExprs arities bindings tokens
        Nothing ->
            let (args,exprs) = splitAt (arities ! name) (parseExprs arities bindings tokens)
            in Call name args : exprs

parseLiteral arities bindings (Zero:tokens) literal =
    parseLiteral arities bindings tokens (False:literal)
parseLiteral arities bindings (One:tokens) literal =
    parseLiteral arities bindings tokens (True:literal)
parseLiteral arities bindings [] literal = [LiteralBits (reverse literal)]
parseLiteral arities bindings [Nil] literal = [LiteralBits (reverse literal)]
parseLiteral arities bindings (Nil:tokens) literal =
    LiteralBits (reverse literal) : parseExprs arities bindings tokens
parseLiteral arities bindings tokens literal =
    LiteralBits (reverse literal) : parseExprs arities bindings tokens

parseBodies :: [Signature] -> Map String [Definition]
parseBodies sigs = parseDefs (groupDefs sigs empty)

arity :: Map String [Definition] -> String -> Int
arity defs fn = case defs ! fn of (Def patterns _):_ -> length patterns

apply :: Map String [Definition] -> String -> [Bits] -> Bits
apply defs fn args =
    let evalMatching Nothing (Def patterns expr) =
            case bind patterns args [] of
                Just bindings -> Just (eval defs bindings expr)
                _ -> Nothing
        evalMatching result _ = result
    in  case foldl evalMatching Nothing (defs ! fn) of
            Just bits -> bits
            _ -> error ("No matching pattern for "++fn)

bind :: [Pattern] -> [Bits] -> [Bits] -> Maybe [Bits]
bind (pattern:patterns) (arg:args) acc =
    case pattern of
        Literal bits ->
            case matchArg bits arg of
                Just [] -> bind patterns args acc
                _ -> Nothing
        Binding bits _ ->                
            case matchArg bits arg of
                Just binding -> bind patterns args (binding:acc)
                _ -> Nothing
        Wildcard bits ->                
            case matchArg bits arg of
                Just _ -> bind patterns args acc
                _ -> Nothing
bind [] [] acc = Just (reverse acc)

matchArg (bit:bits) (argBit:argBits)
    | bit == argBit = matchArg bits argBits
    | otherwise     = Nothing
matchArg [] arg = Just arg
matchArg _ [] = Nothing

eval :: Map String [Definition] -> [Bits] -> Expr -> Bits
eval _ _ (LiteralBits bits) = bits
eval defs bindings (Concat head tail) =
    eval defs bindings head ++ eval defs bindings tail
eval _ bindings (Bound i) = bindings !! i
eval defs bindings (Call fn args) =
    apply defs fn (map (eval defs bindings) args)

main :: IO ()
main = do
    stdinText <- getContents
    interpArgs <- getArgs
    let (files,fn,argFiles) = parseArgs interpArgs []
        stdinBits = textToBits stdinText
    sigs <- readSources files []
    args <- readArgs stdinBits argFiles []
    let defs = parseBodies sigs
    let nils = []:nils
    let result = apply defs fn (take (arity defs fn) (args ++ stdinBits:nils))
    putStr (bitsToStr result)

parseArgs ("-":fn:args) files = (reverse files,fn,args)
parseArgs ["-"] files = (reverse files,defaultFn files,[])
parseArgs (file:args) files = parseArgs args (file:files)
parseArgs [] files = (reverse files,defaultFn files,[])

defaultFn (fn:_) =
    (fst . (break (=='.')) . reverse . fst . (break (=='/')) . reverse) fn

readSources [] sigs = return sigs
readSources (file:files) sigs = do
    text <- readFile file
    let s = sigs ++ ((parseSigs . tokenize) text)
    readSources files s

readArgs _ [] args = return (reverse args)
readArgs stdinBits ("-":files) args =
    readArgs stdinBits files (stdinBits:args)
readArgs stdinBits (file:files) args = do
    text <- readFile file
    readArgs stdinBits files (textToBits text:args)

textToBits text = foldl (++) [] (map charToBits text) where
    charToBits char = map (testBit (ord char)) [7,6..0]

bitsToStr [] = []
bitsToStr bits =
    let (byte,rest) = splitAt 8 bits
        bitToInt flag index = if flag then bit index else 0
        bitsToChar bs = chr (foldl (+) 0 (zipWith bitToInt bs [7,6..0]))
    in
        bitsToChar byte : bitsToStr rest

The demo I got most recently pulled into, involving protocols and file formats that I don't know, as well as working over a weekend, is finished. But now there's another, totally different, demo that needs to be set up by the end of the week.

On another front, the caching optimization that I've also been pulled into is being tested. First, people reviewing the code had problems with

    Object lock = new Object();
    synchronized (lock) {
       ...
    }

because, based on that snippet, they said that the synchronized statement does nothing. However, if they had looked inside the synchronized statement, they would have seen the lock object being put inside a static cache, and, if they had looked at the code just outside of the snippet, they would have seen code extracting stuff from the cache. The snippet is inside the if block that occurs when extracting null from the cache. One of the else blocks would have done a synchronized on a lock object extracted from the cache to block on it instead of duplicating the work done by the thread that created the lock object.

In any case, there was a problem with the code. It seems that the code that generated the stuff in the cache also had some side-effects, so that when there was a cache hit, it would fail due to the side-effects not being done. But the caching code was being blamed, because it didn't fail in the "single-threaded case", which apparently didn't test the cache hit case.

There was a real problem with the caching code that was revealed. If the thread generating the cached data threw an exception, it would release the lock, but the lock object would still be in the cache, so other threads would endlessly loop, waiting for the first thread to replace the lock object with the cached data.

This is what the code should be with the proper clean-up:

for (;;) {
    Object entry;
    synchronized (cache) {
        entry = cache.get(key);
    }
    if (entry == null) {
        Object lock = new Object();
        synchronized (lock) {
            try {
                synchronized (cache) {
                    if (cache.get(key) != null) {
                        lock = null;
                        continue;
                    }
                    cache.put(key, lock);
                }
                CacheData data = computeData(key);
                synchronized (cache) {
                    cache.put(key, data);
                }
                lock = null;
                return data;
            } finally {
                if (lock != null) {
                    synchronized (cache) {
                        cache.remove(key);
                    }
                }
            }
        }
    } else if (entry instanceof CacheData) {
        return (CacheData) entry;
    } else {
        synchronized (entry) {}
    }
}

Note that without the finally block, if computeData() were to throw an exception, all other lookups with that key would go into infinite loops.

The point of this code is to call computeData() only once per key, to avoid load spikes when starting up. Another approach, which was also implemented, is to preload the cache. But that code is much more uninteresting.

Unfortunately, the actual code was made more complicated, though the caching logic should be the same.

In this caching code that I worked on due to a performance problem, I got some feedback proposing to take out a bunch of the synchronized blocks because of some ignorant reasoning that the synchronized blocks don't do anything except hurt performance. My response pointed out why each of the synchronized blocks was necessary, and, for one instance, quoting the javadoc for java.util.HashMap that says, "If multiple threads access a hash map concurrently, and at least one of the threads modifies the map structurally, it must be synchronized externally."

For the synchronized HashMap access, I suppose using java.util.concurrent.ConcurrentHashMap would allow the removal of the synchronization. Since ConcurrentHashMap was introduced in Java 1.5, and the code still had to be Java 1.4, I didn't mention it in my response.

I don't know what will come of this. I don't know the person making the proposal, but I suspect that it is someone who is relatively junior. I hope he or she will learn from this.

There was some new code recently added passing around an extra parameter that did nothing more than stick an extra string in the logging. The reasoning is

...it is pretty much impossible to trace the flow of a single request to the server when other requests are coming in. I propose we add a simple prefix to each log [...] that will contain a unique ID per request that can be grepped in the logs...

But it's actually not impossible to follow a single request, since each log entry includes the name of the thread. But if it helps him, it's fine with me as long as I don't have to maintain that code.

It's like that other almost as silly request from someone who has since left to rename certain threads because he wanted them to have certain names in the logs. Personally, I prefer keeping the thread names from the JVM or from the servlet engine, because they will be unique. I had to rework that thread renaming code after adding some other feature caused thread name collisions.

I noticed an exception handler in a listener that has been added to a service. Why is there a System.exit?

        } catch (IOException e) {
            System.err.println("Could not listen on port: " + portNumber + ".");
            System.exit(-1);
        }

Any IOException in the listener will halt the entire system. I don't know if that is intentional. At the moment, this listener is pretty much a prototype. The rest of the system is running in production. I imagine that testers would be annoyed if the System.exit blocks the testing of everything else if there are problems with the new listener.

Actually, the webapp would probably just respawn repeatedly until the problem fixes itself, which would probably be never.

So I got thrown into this other project that I know almost nothing about, doing stuff with protocols that I know nothing about, and working with data formats that I know nothing about. Fortunately, dealing with the stuff I know nothing about either involves making library calls or porting existing code. Not that making library calls is simple matter when lacking the understanding of what the calls are supposed to do, which means the code winds up being blindly based on sample code.

While porting code that does stuff that I don't understand, and I came across this function:

int GetNextStartCode(Byte_t* pData, size_t bufSize, int startIndex) 
{
    uint32_t dataLen = bufSize;
    uint32_t codeMask = 0x0ffffff;
    uint32_t codeWindow = codeMask;//24 bit window. looking for
    uint32_t startCode = 0x000001;
    for (int i=startIndex; i<dataLen; i++) 
    {
        codeWindow = (codeWindow<<8 | pData[i]) & codeMask;
        if (codeWindow == startCode) 
        {
            assert(i>2);
            return i-3;
        }
    }
    return -1;
}

After simple-mindedly rewriting it in Java, I realized what it was actually doing, and figured it would be clearer to write it like this:

    private int getNextStartCode(byte[] data, int start) {
        for (int i = start; i < data.length-2; i++) {
            if (data[i] == 0 && data[i+1] == 0 && data[i+2] == 1)
                return i;
        }
        return -1;
    }

After looking even more at the code, I realized that the GetNextStartCode was really only being used once with startIndex = 0, and that the subsequent calls were ultimately not used, with bufSize being used in the end.