Solving `Passport Application` with Haskell

https://jameshaydon.github.io/passport/

EngineeringStuf
I've worked on a variety of large UK government systems for the past ten years.

This blog encapsulates the problem of writing government services/software, which often results in strange outcomes.

Writing software for government is essentially the codification of centuries worth of Acts of Parliament.

Now imagine building the HMPO passport system, and then some underlying Law/Act is changed or repealed etc.

Now someone has to find and change everything that the Law/Act affected in all systems.

Now consider that the government frequently outsources this work to expensive consultancies who are motivated to elongate contracts and extract maximum value from the client... And ideally become entrenched.

All whilst building systems of varying quality and inflexibility so that the next time that a Law/Act is changed then this whole process repeats.

There is no central decision making authority to wrangle this problem (there used to be Spend Controls), which is why Government services delivery is so expensive.

pbhjpbhj
Do you have any insight as to why, seemingly, there is no contractual obligation on contractors to make a working system. They seem to make something approximating a working system, vastly overcharge -- like x1000 -- for output that appears to be 3 months of work by one junior programmer, but then get paid as if the system actually worked.
Majromax
> Do you have any insight as to why, seemingly, there is no contractual obligation on contractors to make a working system.

Often, you'll find that procurement is handled by someone working off of an incomplete set of written specifications. The procurement officer probably has no expertise with or informal knowledge of the problem to identify problems in advance, and even if they do legal requirements (ironically designed to ensure fair and transparent procurement) might force them to disregard informal knowledge over the formal specifications.

After the contract is signed, sunk cost fallacies and optimism biases take over. If the project goes over budget it might be easier to cut back on features than admit failure or demand more money, and the project will go over budget once real users start seeing the in-progress work and change requests pile up. Project managers want to be seen managing a successful project; nobody's rewarded for being the first to predict failure.

This is a kind of principal/agent problem that the private sector partially manages with equity or deferred compensation, but the profit motive has a very limited role in government and you'd probably not want to give stock options anyway. Beyond that, even the political overlords have short time horizons of 'the next election', and announcing a grand new project is almost as good as delivering one.

A high-profile example of this problem is the Canadian federal payroll system (https://www.oag-bvg.gc.ca/internet/English/att__e_43045.html), which has led to billions of dollars in fix-ups over a decade or so.

atmosx
Oh dear - you should European Funds for CS research programs. It’s a money making scheme for everyone involved without producing anything of substance and runs for years on end. There even companies “specialized” in every step of the process.
undefeated
It benefits no one to acknowledge that months were spent and the only outcome is millions of dollars wated. Sure, they could try to hold the contractor accountable, but that would put them at risk of being held accountable themselves by their own superiors for hiring that contractor in the first place. So they pretend everything works great, and pay accordingly
atmosx
Has outsourcing ever worked for anyone? I really haven’t heard a success story involving a government outsourcing _something_ like ever.
thrance
I recently renewed my passport in France. There is a new-ish platform here for every document one might want to obtain. I had one form to complete online, that was rather easy to follow. I then had to go to the office once to finish the process. That took only 10 minutes and then the passport was delivered. As painless as can be, I believe.

Whatever the situation is in the UK, it is not hopeless. The government needs to stop being afraid of hiring actual employees and bring all development in-house.

moomin
There was a group of people under the gov.uk banner who were actually really good. But a challenge is that if you employ someone, you need to fit them into your grade structure. And decent programmers earn “Senior Civil Servant” figures.
mgaunard
as a French citizen, I find that surprising.

While renewing a passport is one of the easiest things you can do, I distinctively remember it taking a good half-day (most of which is waiting in line jumping from one desk to another until you find someone that can check all the documents you provided are satisfactory and sign off for you).

Now however, if it's not a renewal but a new application, the difficulty sharply increases.

behnamoh
Haskell has an interesting syntax: it is intuitive after someone explains it to me, but not intuitive much before the explanation.

I don't think it's because I'm used to Algol-based languages (C, Python, etc.). Every Haskell code I've seen is plagued with a plethora of operators which aim to make the code concise but it's not obvious what they do just by looking at them: https://academy.fpblock.com/haskell/tutorial/operators/

tikhonj
At some point, I did a rough count, and the number of operators you encounter in "normal" Haskell code—avoiding lens or domain-specific libraries—was pretty close to the number of operators you'd encounter in, say, JavaScript. This was a while ago and I don't want to redo the exercise now, but, even if we're being generous to JavaScript, practical Haskell needs on the order of 2x as many operators as practical JavaScript, not 10x.

Haskell has some up-front incidental complexity, but it's a difference of degree not kind from popular starting languages. It's easy to underestimate how much you had to learn about Algol-style languages that you've just internalized so well that you don't even realize. I've taught some complete programming beginners (high or middle schoolers) Java and Python and, especially one-on-one, I've seen how they get confused by things I did not even remember required explanation!

For example, in Python, people put : in the wrong place and don't understand how = works. "Basic" syntax like : and = are far more complex than we realize, we've just learned the basics so well that it's second nature. It's similar to how native English speakers don't even realize we have rules for adjective order ("big red ball" vs "red big ball"), while language learners run into it like a brick wall.

IshKebab
> practical Haskell needs on the order of 2x as many operators as practical JavaScript, not 10x.

Yeah but the JavaScript operators are almost all universally understood. There are some exceptions and mistakes, like `===` which shouldn't exist, and `.?` which is new but fairly standard. But most of the operators are just standard maths stuff, normal array/field accesses, etc.

Haskell has normal operators plus a bunch of weird operators that you need to learn. Even if the raw number of operators is only 2x, the number of operators that you have to learn is probably 10x.

Especially when you consider that Javascript operators are fixed so you have to learn them once, whereas Haskell operators are user-defined and Haskell programmers love defining them.

cbsmith
[delayed]
ngruhn
I don't know if you just counted the JavaScript operators but if you count all the instances where some random symbol is used then it's really a ton. There is

    yield*
    function*
    a?.b?.c
    function f(x = 3) {
    x ??= 3
    ...
TypeScript adds even more:

    a!.b
    Type<T>
    type A = B extends C ? D : E
    ...
    
and there are countless ECMA proposals and TypeScript feature request that want to pile on even more. People seem to have zero qualms about cryptic syntax. I think the difference is that this syntax is mostly very ad-hoc. If you know the language already, this just adds a liiiittle bit extra to solve some very specific problem. On the other hand, Haskell operators like <$>, <*>, >>= are extremely general. After years I still learn about new and unexpected ways that you can use them (e.g. what the hell is `fix <$> id`). But paradoxically, because they can be used in so many seemingly unrelated contexts, you have no idea what to use them for at all initially.
skybrian
I think that’s true as far as it goes, but there are further reasons why Haskell is more difficult. Here is one:

In Haskell, the lack of parentheses for function calls plus currying means that to read a function call, you need to already know how many arguments the function takes, which I feel adds a new level of difficulty over languages where you can often guess what a function does based on its name, without looking it up.

As a result, often Haskell reads more like math, where without knowing each symbol’s definition, you’re lost.

I’ve seen cryptic JavaScript too, but less often.

jameshh
> the lack of parentheses for function calls plus currying means that to read a function call, you need to already know how many arguments the function takes

While I agree with the general sentiment of what you are saying, note that the syntax has nothing to do with it, it is purely about Haskell using currying excessively. The syntactic translation between Haskell and JS is straight-forward and 1-1:

    f x y z      ->  f(x)(y)(z)
    f x (y,z) w  ->  f(x)(y,z)(w)
I agree that excessive currying is not great, and generally push for non-curried arguments unless a curried form realy is used in practice. But for this to really be comfortable, and to still enjoy all the hgiher-order programming that's nice with Haskell, we would need good records (strucural, anonymous, extensible), which it doesn't really have right now, so we are stuck with currying.
theLiminator
> it is intuitive after someone explains it to me, but not intuitive much before the explanation

Isn't that pretty close to what something being intuitive means? Ie. something being intuitive means that it's easily understood or learned without requiring prior knowledge or instruction.

gylterud
With Haskell code it is often so that the code which actually does something is expressed in terms of combinators. These combinators are often very general, so one must do a bit of mental work to understand what they do in this particular context.

Once you understand what the combinators do in a particular context the code often reads very easily, and what it does is very intuitive and natural.

Best example I can think of is applicative parsers, where there are scattered <*> and <$> operators around, but if you just ignore them the parser just looks like a very intuitive description of what the expressions you want to parse look like.

mejutoco
I think Larry Wall mentioned something like this regarding perl. Things can only be intuitive after learning them. I do not think it is syntax you are complaining about either. Some constructs in Haskell reference a certain model (monads, lenses) and no syntax is going to infuse any reader with that knowledge if they do not possess it already.
the_af
Most of the operators in that link are bog standard, and among the least interesting things to say about Haskell.

If I didn't know programming and picked at random some Java program, I wouldn't understand much either. If I didn't take a single math course and I tried to read a math formula, I wouldn't understand the operators or anything either.

The standard should be: how much can you understand once you learn the basics and do a couple of tutorials?

munchler
I love this. It starts as a parody, then slowly morphs into a serious and elegant solution to the "game". By the end, I almost wish I could play it myself. Bravo.
Prcmaker
I think you can play, just much harder to get through the end game stage.
returningfory2
Anyone can play but only ~70 million people can win :)
cherryteastain
> What are the base cases? A base case is an ancestor whose Britishness does not depend on a parent, e.g. someone who was naturalised, or born in the UK before 1983 (which makes one unconditionally british regardless of parents). That's right, further into the future we get from 1983, the taller these call-stacks can get.

This is insane to me, coming from a country that simply has a central database of all citizens and foreign residents, and a citizenship check is probably something like a single SELECT on an SQL table.

mcsniff
Not only can you complete a UK Passport Application completely online (save for mailing any required documentations to HMPO), you can apply for a brand new passport as a "new" citizen (by adoption, naturalisation, or descent), online, from outside the UK, with just a mobile device without downloading some special app, including taking the photos.

Sure, you can play it on "hard mode " and do it with paper and pen for the lulz, but my experience was extremely efficient, fast, and straight-forward.

whitehexagon
I went through the UK online passport renewal system last year, and it is the best website I have used in a very very long time. A real breath of fresh air. It worked perfectly on my old computer and 'ancient' browser. Everything very simple / accessible / clear, colours, text, buttons, fields, requirements. A simple step-by-step navigation and actually friendly.

It is a shame that more businesses dont care about their customers enough to invest in such a well designed website. Well done HMPO.

icheyne
I work with Equal Experts, who built this system for HMPO. Feels good to get this feedback.
jameshh
I agree, I also renewed my passport recently and the website and process is quite smooth. It's the more complex cases, first requests from abroad, etc., that lack automation.
vixen99
Seconded! In my experience, Gov.UK does pretty well in general and am happy to say that as one who privately fumes about various aspects of Britain in recent years.
digianarchist
Renewed recently which was my first time using the online system. The entire process was exceptionally fast. I got the passport back in less than 2 weeks from California.

In comparison my Canadian passport renewal (damaged after 3 years) from California took 4 months. Was entirely paper based and had ridiculous requirements such as requiring a reference and a photograph stamped by photographer.

I know there's a digital pilot that's ongoing. That should be rolled out ASAP.

kalleboo
Just being able to do it via mail sounds like luxury, to renew a Swedish passport you need to visit an embassy in person. And then in 2 weeks go back in person to pick it up. Really fun when the embassy is on the other side of the country and it means expensive plane trips. This was even more of a problem during COVID when borders were closed, since Sweden doesn't have embassies in every single country.

And they expire in 5 years, not 10 like many other countries.

OptionOfT
I need to renew my Belgian passport soon, and I live in the United States.

Even though the consulate has my biometric data, I need to visit them in person.

My only saving grace is that they sometimes visit my city., and I can register there. But that registration is only valid for 1 year, and they don't visit every year...

And that is still 2h+ one way of travel for me, for something they already have.

belgiandudette
The Belgian passport comes in exceptionally fast though. For me it was ready for pick up in the Belgian embassy in less than a week from application.
new299
For the benefit of anyone else doing this in Japan, these are the documents I had to submit. It will be different depending on circumstance, in particular I think the author may also be born outside the UK which may require other/more documents.

Anyway I had to submit:

Certified copy of my birth certificate, order from the UK general register office.

Original (not photocopy) of Japanese family register and translation.

Certificate of acceptance of notification of birth, original and translation.

Original Marriage certificate and translation.

Colour photocopy of every page of my childs Japanese passport.

Copy photocopy of the passport of an American or British citizen who confirms that child is mine.

The process is pretty unclear, and in general you seem to have to just keep submitting documents until they are satisfied.

jameshh
> I think the author may also be born outside the UK which may require other/more documents

In fact `applicant's father` (me) was born in the UK in this case, but `applicant's father's father` was not, the cause of the extra complications.

new299
:(

Weirdly this never came up for me (and I'm in a similar situation). I suspect what documents are ask for varies a lot by who is assessing the application.

Hope it all works out!

crooked-v
I have to wonder how they would handle cases where a person doesn't have a legal father at all.
lmm
If you don't have a legal father then you can't claim British citizenship through your father. That's all pretty straightforward. (Although there's a fun case where you can claim citizenship by double descent because your ancestor wasn't able to claim citizenship because of sexist historical laws).
d1sxeyes
That’s interesting, no requirement for parental birth certificates?

I was asked to provide (I am British, wife is Hungarian, living in Hungary) both parents’ birth certificates and all four grandparents’ birth certificates.

My wife is not in touch with her father, so we had to submit a cover letter explaining the situation, which was accepted (although I don’t really see why it was relevant anyway, my son’s Britishness is derived from my Britishness, her nationality and the nationality of her parents is essentially irrelevant).

We also didn’t have to copy the passport of anyone else, but we did have to have a family friend do the confirmation online (family friend is a lawyer in the UK, so was on the list of approved jobs for this).

jameshh
> although I don’t really see why it was relevant anyway, my son’s Britishness is derived from my Britishness, her nationality and the nationality of her parents is essentially irrelevant

This is essentially happened to me, the "75% of the initial doc requests turned out to be irrelevant" was referring to this sort of thing.

new299
Parent of applicant but not grandparents, at least in my case.
Cyph0n
> Colour photocopy of every page of my childs Japanese passport.

But why?

enqk
Another reason is that the UK forbids the child to have different names on their japanese passport and their UK passport, which is often (almost always?) the case
davchana
Don't know in this case, but some other countries ask for it to deduce your travel history (any current passport itself goes to them, with color copies of every page of old passports). Qatar wanted it because I shared name with somebody born 40 years before me.
Cyph0n
Interesting. Just out of curiosity: did you naturalize as a Qatari (didn’t even know it was possible!), or was it a visa-related thing?
pbhjpbhj
I too have enjoyed a game of *Passport Application* by proxy.

One of the interesting rules the OP doesn't appear to mention is the 'mutable history' rule. Herein, the facts contained in a single document can alter over even short periods of time requiring that a document, already scanned into the Examiners system of fact recording, can be required to be returned for subsequent scans. Verifying documents have not altered in their facts is, some say, tacit acknowledgement by the Examiner breed that a multiverse exists.

The communication channel between the "NPC" telephone agents and the Examiners appears to be somewhat akin to prayer, only vague inferences can truly be received.

Aside, I suspect the OP maybe be a Mornington Crescent aficianado; a game seemingly similar in rule-topology to the aforementioned Passport Application.

tanh
Yes and in the British system names can be fluid, changed at will with no declarations being made. That’s how my mother was able to register my birth with the surname of her mother’s third (!) marriage. Birth certificates are also immutable in certain ways but not entirely everything on it is immutable.

Not to mention the passport office has forgotten previous decisions for my own passport where this sort of surname discrepancy has been explained away. (Including one passport where I am listed on a page of her passport!)

The NPCs are infuriating. I called, emailed and received a different answer every time. It was like a poor LLM.

After applying I had to explain to my mother that within the same batch of applications, one daughter required additional UK documents but the other one didn’t (the passport came), purely because one was born in the UK and the other one wasn’t. She rightly flipped out, raised it with an MP and it was resolved within a day.

pbhjpbhj
Similarly for us, siblings each required different treatment. Even when part of the same application. All born in the same UK hospital to the same parents and having the same home address. Fun.

It was interesting to speculate what exactly the one child whose passport came months later had done that warranted the Examiner's extra scrutiny. Something to do with reincarnation I expect.

vishnugupta
This reminds me of an article read about how DSL was used to automate/encode Dutch tax code.

I can’t recall the article I read but this is a good presentation.

https://tomassetti.me/wp-content/uploads/2020/10/2018-04-10-...

tanh
I just went through hell this with one of my daughters. She was born in Japan and I had a hard time getting my mother’s surname matching what was on my birth certificate.

Long story short, I demonstrated I was born in England and that I have citizenship (a passport) so no matter the possibility, it must be something that can be passed on. That doesn’t match the guidelines of documents required but it’s much less of a pain I reckon.

throwawayben
I played this a few years ago.

Despite being born in the UK to a British father my character wasn't previously British due to parents being unmarried and being born after 1983 but before the 2006 rules update.

I'm not totally sure when but I think some time in the 2010s the 2006 update became retroactive so my character was allowed to play. Didn't have to do too many side quests - the main one was retrieving father's birth certificate.

My character also had to attend the endgame ceremony and swear allegiance to the queen, which seemed odd for someone born in the country and lived here all his life

jameshh
At least you got the bonus endscreen credits!