2018-11-13

So, I finished my Haskell web project...

I thought it would be mostly a Haskell project, but it's turned out to involve a lot of Javascript, too - which is good in that I now have a project that might motivate me to learn more about the frontend side of things.

~/code/whereami
(0) <hephaestus:kyle> $ cloc app assets static src 
       6 text files.
       6 unique files.                              
       0 files ignored.

github.com/AlDanial/cloc v 1.74  T=0.21 s (28.0 files/s, 1874.2 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Haskell                          4             53             11            219
JavaScript                       1             15              5             80
HTML                             1              1              0             18
-------------------------------------------------------------------------------
SUM:                             6             69             16            317
-------------------------------------------------------------------------------

There are a lot of further improvements that I could make; it's certainly not a done project, but it is in a state where I am using it, even if only to submit location data. It's deployed to my Hetzner colo machine via docker-compose.

One of the things that sort of held me back is that I don't have any testing code. When I'd make changes, I'd ran a local dev server and try to use the app. Debugging was often painful, because I would just get a log message about that the server returned a 500. I'd like to investigate more into this. However, while testing would have been useful, it wouldn't have covered everything. For example, the Javascript code wasn't always sending the full fields when I thought it was, which masked some errors. I guess maybe Quickcheck might have helped with this, but I don't think so - the parser operates on JSON strings, and my constructor requires all the fields to be in place. It would help a lot, though, so it's something I'd like to learn more about and figure out how to do.

Another thing that testing the Haskell wouldn't have helped with was the hour or two I spent debugging a CORS issue. In my Javascript, I'd hardcoded a URL to 'http://localhost:4000/' - I know hardcoding URLs is a bad thing, but I didn't yet know about window.location and its components. In any case, I forgot that's what the URL was, and I just kept getting CORS errors that weren't really useful to debug. Maybe it's that I'm not a frontend dev, and wasn't terribly familiar with the debug tools, but either way, this was one of the most challenging things to debug - the Haskell was actually easier... I spent a bunch of time on a wrong path, thinking it was an issue with the reverse proxy setup, and I tried all manner of headers and header tweaking. Turns out it wasn't that :) I eventually figured it out by noticing that the server logs weren't showing any POSTs.

Another interesting thing that I found out is that the SQLite library that I'm using parses the TIMESTAMP type as a string, which made getting it with my Coordinate type (where the timestamp field is an Int) impossible. That required a schema change from timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP to INTEGER NOT NULL. This also encodes the decision to require a timestamp for the coordinate; to do so, I wrote a quick filter function that would add the current timestamp to coordiantes if the timestamp was 0.

This, actually, turned out to be probably the most difficult Haskell bug to track down. What I hadn't realised about Haskell is that it won't warn on inexhaustive case matches, and it compiled fast enough on my machine that I missed the warning message in the Stack output. I fixed this by adding

  ghc-options:
  - -fwarn-incomplete-patterns
  - -fwarn-incomplete-uni-patterns
  - -Werror

to package.yaml, but Scotty was covering the actual error and converting it into a 500. This was another source of trouble: rather than have the app explode with an error message (or even printing a useful error message), all I saw were error 500s. I had to putStrLn debug it... there's probably a way to get these error messages out, but I couldn't figure it out. Eventually, I ran an upgrade on the colo machine, which is pretty slow - slow enough that I saw the GHC warning and was able to pinpoint the problem. Still, it took longer than I'd like and admittedly, unit testing would have caught this.

Deploying was a bit of a pain until I switched Docker to running stack build on the bindmounted read-write directory. I'm sure there's all kinds of issues with this, but I'm relying on basic auth and TLS to protect things for now. Eventually, I'll want to build an Argon2-based session system using a username and password and maybe even U2F or a TOTP. I need to figure out what that's going to look like for this.

I also was able to excise all of the calls to unsafePerformIO by doing things at the top-level: reading environment variables for configs, opening a connection to the database, etc. It was a shift in how I was thinking about things and a far cry from the part where I used a globally-defined IORef or STM TVar. I do think I understand better how I would use those (e.g. instantiating in main and passing them off to functions).

On a web-dev note, I made the mistake of confusing the Content-Type and Accept headers: Content-Type is what the server returns, and Accept is what the client requests. I was using Content-Type for both purposes for a while and wondering why it wasn't working. I think I figured it out by looking at the headers of a request.

Finally, on a meta note, I spent a lot of time programming this on my docker T480. I've got an old Acer 24" 1080p monitor, a full-size WASD CODE keyboard (with MX Cherry Clear switches), a wireless Logitech trackball, and a pair of Sennheiser HD650s plugged into a Lenovo Ultra docking station, and this setup has proven to be quite productive. It's rather easy to just lift my laptop off the docking station to take it elsewhere (e.g. the living room) when I need to move around, and I love having a mechanical keyboard. I might upgrade to 4K monitor later, but I'm getting along just fine with the smaller one. Both my laptop and the monitor have the same resolution (1920x1080) so there's no weird resizing of things during the handoff.

It's nice to be writing software for myself again.


Tags: ,