2024-05-20

• Learned that when “upserting” data in Postgres, the ON CONFLICT DO NOTHING clause will not return any rows when it triggers. A solution (that I’ve verified works) can be found here.

• Thinking about how to package the software components into something repeatable and easily-installable. FPM-created DEB packages might be the answer, containing the services and their systemd files?

• Tried using ore as a Dependency Injection container. I like its API, but I don’t think DI gels well with my more-explicit coding style. I worry a lot about spooky action at a distance.

• Refactored the bin/web/components functions to be more explicit about what they’re operating on.

  • Relatedly, thinking about how to package these as HTML Custom Components without involving a bunch of JS nonsense?
  • Also, how can I make it possible for components to add their own static files to the HTTP server (for things like icons, or stylesheets)?

• Thinking about whether I want to include the raw cert bytes in the database for lib/data/cert.Cert.

  • Pros: can come along later and re-parse them to draw out additional information that we missed, makes executing checks post-hoc easier.
  • Cons: larger data storage requirements, especially when we start implementing Certificate Transparency monitoring.
  • At this time, I’m going to go ahead with it. We can always pare back later if we find it’s too much, but I need to remember that I have a lot of resources on the production server.

• Working on the above, I learned that encoding/json (and similar packages that use reflect) can’t reach into unexported fields. That makes things a little awkward as far as hiding data, but I can be careful about what I use.

• I love Zed for many reasons- the editor ergonomics of it seem much nicer somehow- but I do find it frustrating how little extra functionality it has. Examples include: Markdown previews (with Mermaid support) and TODO highlighting. For now, I will keep using VS Code as my editor as it has those features.

• Thinking about what I want my work machine for Callisto to be- might be worth it to pick up a Mac Mini and decommission my old Intel Macbook from BlockFi. I want an ARM machine, for sure, but it’s difficult to find an affordable 16GB option, which I consider the minimum for development work.

• Interesting finding for Postgres queries: I needed a way to find “of all the certs located for this scan, what one doesn’t have any chain_cert_id pointing to it?” The query that ended up working was:

  SELECT id, chain_cert_id, issuer, valid_domains FROM data_certs c WHERE NOT EXISTS (SELECT 1 FROM data_certs WHERE c.id = chain_cert_id);
  

  The NOT EXISTS part is the most important there.

• Updated the scans to properly manage and display the certs that they find.

• For now, I’m going to keep my existing push mechanisms that I wrote with Fabric, since they allow me to do more sysadmin tasks during a push.

  • Worth translating this small library into Go?

• Cross-compilation

  • How to handle this with Please? It would be real nice to make the push script depend on built artifacts, to properly leverage the build system.

  • Please’s Docker build rules have some interesting ideas in them- the problem I see is that they only produce the scripts that WOULD run the builds, not the artifacts themselves. Most of this is from the assumption that they are building containers- I am not.

  • It appears that Please actually has a built-in cross-compilation option- --arch in plz build works. I wonder if I can make plz push use this automatically.

    • Yes, this is possible- I can pass the option in the alias config section.

    • Also, are there potential issues from cross-compilation like this? DNS lookups, for instance?