danielquinn

joined 1 year ago
[–] [email protected] 2 points 1 day ago

Canadaland recently did an episode on this very subject and as a once-supporter of these sites, I found it eye-opening.

[–] [email protected] 14 points 2 days ago* (last edited 2 days ago) (2 children)

Oh boy are you going to love-to-hate this then. It's best viewed on a proper computer, but you'll get the gist on mobile too.

[–] [email protected] 5 points 3 days ago (2 children)

To be clear, I'm not throwing shade. That's an impressive piece of software. It's just, given the number of stories I've heard (and experienced) about Bash's tricky syntax leading to Bad Things, I'm less comfortable with running this than I would be with something in a language with fewer pitfalls.

But if others take the chance and it sticks around a bit, I'll come around ;-)

Thanks for the contribution! It's a great idea, and with Google fucking about with blocking things like NewPipe, a project like this is a great answer to that.

[–] [email protected] 6 points 3 days ago* (last edited 3 days ago) (4 children)

That looks really impressive, but at nearly 1000 lines of Bash, I'm afraid I'm not comfortable running it on my machine. My Bash-foo isn't strong enough to be sure that there isn't a typo in there that could nuke my home folder.

[–] [email protected] 1 points 4 days ago

But there's nothing stopping you from loading realistic (or even real) data into a system like this. They're entirely different concepts. Indeed, I've loaded gigabytes of production data into systems similar to what I'm proposing here (taking all necessary precautions of course). At one company, I even built a system that pulled production into a developer-friendly snapshot while simultaneously pseudo-anonymising that data so it can be safely (for some value of ${safe}) be tinkered with in development.

In fact, adhering to a system like this makes such things easier, since you don't have to make any concessions to "this is how we do it in development". You just pull a snapshot from the environment you want to work with and load it into your Compose session.

[–] [email protected] 1 points 4 days ago (2 children)

It sounds like you're confusing the application with the data. Nothing in this model requires the use of production data.

[–] [email protected] 2 points 6 days ago

I feel like you must have read an entirely different post, which must be a failing in my writing.

I would never condone baking secrets into a compose file, which is why the values in compose.yaml aren't secrets. The idea is that your compose file is used exclusively for testing and development, where the data isn't real, and the priority is easing development. When you deploy, you don't use that compose file because your environment is populated by whatever you use in production (typically Kubernetes these days).

You should not store your development database password in a .env file because it's not a secret. The AWS keys listed in the compose are meant to be exactly as they are there: XXX, because LocalStack doesn't care what these values are, only that they exist.

As for the CLI thing, again I think you've missed the point. The idea is to start from a position of "I'm building images" and therefore neve have a "local app, (Django, sqlite)" because sqlite should not be used unless that's what's used in production. There should be little to no difference between development and production, so scripting a bridge between these doesn't make a lot of sense to me.

[–] [email protected] 1 points 6 days ago (2 children)

I don't mean to be snarky, but I feel like you didn't actually read the post 'cause pretty much everything you've suggested is the opposite of what I was trying to say.

  • A CLI to make things simple sounds nice, but given that the whole idea is to harmonise the develop/test/deploy process, writing a whole program to hide the differences is counterproductive.
  • Config settings should be hard-coded into your docker-compose file and absolutely not stored in .json or .env files. The litmus test here is: "How many steps does it take to get this project running?" If it's more than 1 (docker compose up) it's too many.
  • Suggesting that one package Django into a single Lambda seems like an odd take on a post about Docker.
[–] [email protected] 2 points 6 days ago* (last edited 6 days ago) (1 children)

High praise! Just keep in mind that my blog is a mixed bag of topics. A little code, lots of politics, and some random stuff to boot.

[–] [email protected] 9 points 6 days ago (1 children)

It's a tough one, but there are a few options.

For AWS, my favourite one is LocalStack, a Docker image that you can stand up like any other service and then tell it to emulate common AWS services: S3, Lamda, etc. They claim to support 80 different services which is... nuts. They've got a strange licensing model though, which last time I used it meant that they support some of the more common services for free, but if you want more you gotta pay... and they aren't cheap. I don't know if anything like this exists for Azure.

The next-best choice is to use a stand-in. Many cloud services are just managed+branded Free software projects. RDS is either PostgreSQL or MySQL, ElastiCache is just Redis, etc. For these, you can just stand up a copy of the actual service and since the APIs are identical, you should be fine. Where it gets tricky is when the cloud provider has messed with the API or added functionality that doesn't exist elsewhere. SQS for example is kind of like RabbitMQ but not.

In those cases, it's a question of how your application interacts with this service. If it's by way of an external package (say Celery to SQS for example), then using RabbitMQ locally and SQS in production is probably fine because it's Celery that's managing the distinction and not you. They've done the work of testing compatibility, so theoretically you don't have to.

If however your application is the kind of thing that interacts with this service on a low level, opening a direct connection and speaking its protocol yourself, that's probably not a good idea.

That leaves the third option, which isn't great, but I've done it and it's not so bad: use the cloud service in development. Normally this is done by having separate services spun up per user or even with a role account. When your app writes to an S3 bucket locally, it's actually writing to a real bucket called companyname-username-projectbucket. With tools like Terraform, the fiddly process of setting all this up can be drastically simplified, so it's not so bad -- just make sure that the developers are aware of the fact that their actions can incur costs is all.

If none of the above are suitable, then it's probably time to stub out the service and then rely more heavily on a QA or staging environment that's better reflective of production.

59
Developing with Docker (danielquinn.org)
submitted 1 week ago* (last edited 5 days ago) by [email protected] to c/[email protected]
 

I've been writing code professionally for 24 years, 15 of which has been Python and 9 years of that with Docker. I got tired of running into the same complications every time I started a new job, so I wrote this. Maybe you'll find it useful, or it could even start a conversation, but this post has been a long time coming.

Update: I had a few requests for a demo repo as a companion to this post, so I wrote one today. It includes a very small Django demo user Docker, Compose, and GitLab CI.

[–] [email protected] 2 points 1 week ago (2 children)

You got the eyes just right!

[–] [email protected] -4 points 1 week ago

It really doesn't. See my other comment here for more detail.

 

...so I found out how to fix it

 

It would seem that I have far too much time on my hands. After the post about a Star Trek "test", I started wondering if there could be any data to back it up and... well here we go:

Those Old Scientists

Name Total Lines Percentage of Lines
KIRK 8257 32.89
SPOCK 3985 15.87
MCCOY 2334 9.3
SCOTT 912 3.63
SULU 634 2.53
UHURA 575 2.29
CHEKOV 417 1.66

The Next Generation

Name Total Lines Percentage of Lines
PICARD 11175 20.16
RIKER 6453 11.64
DATA 5599 10.1
LAFORGE 3843 6.93
WORF 3402 6.14
TROI 2992 5.4
CRUSHER 2833 5.11
WESLEY 1285 2.32

Deep Space Nine

Name Total Lines Percentage of Lines
SISKO 8073 13.0
KIRA 5112 8.23
BASHIR 4836 7.79
O'BRIEN 4540 7.31
ODO 4509 7.26
QUARK 4331 6.98
DAX 3559 5.73
WORF 1976 3.18
JAKE 1434 2.31
GARAK 1420 2.29
NOG 1247 2.01
ROM 1172 1.89
DUKAT 1091 1.76
EZRI 953 1.53

Voyager

Name Total Lines Percentage of Lines
JANEWAY 10238 17.7
CHAKOTAY 5066 8.76
EMH 4823 8.34
PARIS 4416 7.63
TUVOK 3993 6.9
KIM 3801 6.57
TORRES 3733 6.45
SEVEN 3527 6.1
NEELIX 2887 4.99
KES 1189 2.06

Enterprise

Name Total Lines Percentage of Lines
ARCHER 6959 24.52
T'POL 3715 13.09
TUCKER 3610 12.72
REED 2083 7.34
PHLOX 1621 5.71
HOSHI 1313 4.63
TRAVIS 1087 3.83
SHRAN 358 1.26

Discovery

Important Note: As the source material is incomplete for Discovery, the following table only includes line counts from seasons 1 and 4 along with a single episode of season 2.

Name Total Lines Percentage of Lines
BURNHAM 2162 22.92
SARU 773 8.2
BOOK 586 6.21
STAMETS 513 5.44
TILLY 488 5.17
LORCA 471 4.99
TARKA 313 3.32
TYLER 300 3.18
GEORGIOU 279 2.96
CULBER 267 2.83
RILLAK 205 2.17
DETMER 186 1.97
OWOSEKUN 169 1.79
ADIRA 154 1.63
COMPUTER 152 1.61
ZORA 151 1.6
VANCE 101 1.07
CORNWELL 101 1.07
SAREK 100 1.06
T'RINA 96 1.02

If anyone is interested, here's the (rather hurried, don't judge me) Python used:

#!/usr/bin/env python

#
# This script assumes that you've already downloaded all the episode lines from
# the fantastic chakoteya.net:
#
# wget --accept=html,htm --relative --wait=2 --include-directories=/STDisco17/ http://www.chakoteya.net/STDisco17/episodes.html -m
# wget --accept=html,htm --relative --wait=2 --include-directories=/Enterprise/ http://www.chakoteya.net/Enterprise/episodes.htm -m
# wget --accept=html,htm --relative --wait=2 --include-directories=/Voyager/ http://www.chakoteya.net/Voyager/episode_listing.htm -m
# wget --accept=html,htm --relative --wait=2 --include-directories=/DS9/ http://www.chakoteya.net/DS9/episodes.htm -m
# wget --accept=html,htm --relative --wait=2 --include-directories=/NextGen/ http://www.chakoteya.net/NextGen/episodes.htm -m
# wget --accept=html,htm --relative --wait=2 --include-directories=/StarTrek/ http://www.chakoteya.net/StarTrek/episodes.htm -m
#
# Then you'll probably have to convert the following files to UTF-8 as they
# differ from the rest:
#
# * Voyager/709.htm
# * Voyager/515.htm
# * Voyager/416.htm
# * Enterprise/41.htm
#

import re
from collections import defaultdict
from pathlib import Path

EPISODE_REGEX = re.compile(r"^\d+\.html?$")
LINE_REGEX = re.compile(r"^(?P<name>[A-Z']+): ")

EPISODES = Path("www.chakoteya.net")
DISCO = EPISODES / "STDisco17"
ENT = EPISODES / "Enterprise"
TNG = EPISODES / "NextGen"
TOS = EPISODES / "StarTrek"
DS9 = EPISODES / "DS9"
VOY = EPISODES / "Voyager"

NAMES = {
    TOS.name: "Those Old Scientists",
    TNG.name: "The Next Generation",
    DS9.name: "Deep Space Nine",
    VOY.name: "Voyager",
    ENT.name: "Enterprise",
    DISCO.name: "Discovery",
}


class CharacterLines:
    def __init__(self, path: Path) -> None:
        self.path = path
        self.line_count = defaultdict(int)

    def collect(self) -> None:
        for episode in self.path.glob("*.htm*"):
            if EPISODE_REGEX.match(episode.name):
                for line in episode.read_text().split("\n"):
                    if m := LINE_REGEX.match(line):
                        self.line_count[m.group("name")] += 1

    @property
    def as_tablular_data(self) -> tuple[tuple[str, int, float], ...]:
        total = sum(self.line_count.values())
        r = []
        for k, v in self.line_count.items():
            percentage = round(v * 100 / total, 2)
            if percentage > 1:
                r.append((str(k), v, percentage))
        return tuple(reversed(sorted(r, key=lambda _: _[2])))

    def render(self) -> None:
        print(f"\n\n# {NAMES[self.path.name]}\n")
        print("| Name             | Total Lines | Percentage of Lines |")
        print("| ---------------- | :---------: | ------------------: |")
        for character, total, pct in self.as_tablular_data:
            print(f"| {character:16} | {total:11} | {pct:19} |")


if __name__ == "__main__":
    for series in (TOS, TNG, DS9, VOY, ENT, DISCO):
        counter = CharacterLines(series)
        counter.collect()
        counter.render()
 

My father is 75 and not very capable on a computer. He's got an old MacBook Air at home behind a typical ISP router for which he has no access controls (so no port forwarding).

My immediate need is actually not his machine at all, but the Raspberry Pi I installed at his house before I left the country and forgot to enable cron on so it's not doing what I need yet. However, it would be really nice if I could also do one of the following as well:

  • VNC (or something) into his computer whenever something "isn't working" rather than doing the talk-him-through-it dance over Skype.
  • Install a new OS (the Mac is no longer supported by MacOS). I don't know how plausible this is though.

My current plan is to email him a shell script that should create a reverse SSH tunnel to a server in Montréal or something and then I can shell into his Mac through there. It's not ideal though since we're still talking shell scripts and he's easily frustrated.

I know that in Windows land there are all sorts of tools scammers use to take over a machine remotely. Does Mac allow for the same thing? Note that I only have Linux machines available to me on this side of the Atlantic.

 

I'm working on a some materials for a class wherein I'll be teaching some young, wide-eyed Windows nerds about Linux and we're including a section we're calling "foot guns". Basically it's ways you might shoot yourself in the foot while meddling with your newfound Linux powers.

I've got the usual forgetting the . in lines like this:

$ rm -rf ./bin

As well as a bunch of other fun stories like that one time I mounted my Linux home folder into my Windows machine, forgot I did that, then deleted a parent folder.

You know, the war stories.

Tell me yours. I wanna share your mistakes so that they can learn from them.

Fun (?) side note: somehow, my entire ${HOME}/projects folder has been deleted like... just now, and I have no idea how it happened. I may have a terrible new story to add if I figure it out.

 

I've got a very simple Kodi setup:

  • Arch Linux on a laptop behind the TV
  • Media files on a server upstairs, shared over NFS

I've been running Kodi quite successfully on this machine for years, but with the Omega update, videos play without audio for about 10seconds, then freeze. Sometimes if I wait a while, I see subtitles for the episode while the video is frozen. Music doesn't play either. The interface freezes too, to the point where I have to kill -9 it. Switching from Wayland to Xorg hasn't had an effect.

I tried deleting ~/.kodi and restarting, but nothing changes.

Has anyone else run into this?

 

A break from the usual in this community, but I trust it'll be appreciated. I think this is very solarpunk: using technology to improve the lives of all creatures.

 

I've been playing a lot of Fallout 4 over the holidays. I started and finished the Nuka World DLC (killed all the baddies), made it to level 90, etc.

Today I was playing on my Deck as the battery got a little low (11%) so I saved my game, exited the game, and went to shut down.

As it was shutting down, the Deck displayed a message, something like "Syncing to Steam Cloud" as the logo was spinning.

A few hours later, on a full charge, I booted it back up, started Fallout 4 again and... some of my old saves are there, but only about 30% of them, and critically not the most recent ones.

Has this ever happened to anyone else? Is this a known issue? Can I fix it, or report it? I've basically lost interest in finishing the game now.

 

His original post , titled I can't sleep, is some brilliant writing. When we talk about the chilling effect that criticism of Israel creates in industries everywhere (including ours) this is what that looks like.

 

I needed something for a presentation I'm doing on advanced Linux, so I thought something like this might be appropriate.

Annoyingly, I can't seem to get Bing to generate an image that isn't square.

view more: next ›