pcouy

joined 1 year ago
[–] [email protected] 1 points 3 weeks ago

I was exited to read about the recent surge of brute force attempts I received from IPs my fail2ban has not previously seen, but this is just a generic piece from 6 months ago :(

 

I've kept playing with shader programming and managed to export a trained neural network's weights as GLSL variable definitions. The code is ugly as hell as I've done a lot of quick experiments with it, and I went all-in with macros where functions would probably be better suited. I hope you still find it interesting.

Excluding neural network weights, the whole thing is ~300 lines of code and can run a few variations of a simple convolutional network.

[–] [email protected] 1 points 1 month ago

Thank you for the link. I've seen it posted a few days ago.

The caching proxy for this tutorial should easily work with any tile server, including self-hosted. However, I'm not sure what the benefits would be if you are already self-hosting a tile server.

Lastly, the self-hosting documentation for OpenFreeMap mentions a 300GB of storage + 4GB of RAM requirement just for serving the tiles, which is still more than I can spare

[–] [email protected] 2 points 2 months ago (1 children)

Hey ! I (superficially) looked up the Belousov-Zhabotinsky reaction and can confirm it is related to the Gray-Scott model.

The Belousov-Zhabotinsky reaction involves an autocatalytic reaction and chemical species diffusing at different rates, just like in the Gray-Scott model. The main differences are related to constraints of doing actual chemistry instead of simulating it :

  • The speed constant is (roughly) fixed for any given reaction (and temperature). Scientists cannot tune speed constants like I did in the simulation
  • In the simulation, we constantly add some "food" and remove some catalyst. In an actual chemical reactor, there must be an process to achieve this. A real world implementation of the Gray-Scott model would probably use something like a semi-permeable membrane above/below the petri dish. In the Belousov-Zhabotinsky reaction, it is other chemical reactions that ensure "food" gets replenished and the catalyst gets consumed
[–] [email protected] 1 points 2 months ago (1 children)

I did not expect such a detailed code review (the fact that you wrote it on mobile impresses me even more), but I strongly agree with everything you mentioned. I think I was so caught up learning GLSL and its quirks, then playing and experimenting with the simulation, that I "forgot" my coding standards. Anyway, I'll make sure to take some time to update both the code and the article following your recommandations.

[–] [email protected] 3 points 2 months ago

I'll make sure to look into it and tell you what the differences are. The patterns definetly look similar to the "wave" or "spirals" preset from my video

 

This is a simulation of the Gray-Scott reaction-diffusion model running on the GPU. In such systems, an auto-catalytic reaction involving two chemical species is happenning concurrently with diffusion. Despite the apparent simplicity of the model, simulating it with cherry-picked sets of parameters produces a wide range of emerging behaviors.

 

cross-posted from: https://lemmy.pierre-couy.fr/post/678825

Hi ! I've been working on this article for the past few days. It would mean a lot to me if you could provide some feedback.

It is about implementing a physico-chemical simulation as my first attempt to write a shader. The code is surprisingly simple and short (less than 100 lines). The "Prerequisite" and "Update rules" sections, however, may need some adjustments to make them clearer.

Thanks for reading

 

cross-posted from: https://lemmy.pierre-couy.fr/post/678825

Hi ! I've been working on this article for the past few days. It would mean a lot to me if you could provide some feedback.

It is about implementing a physico-chemical simulation as my first attempt to write a shader. The code is surprisingly simple and short (less than 100 lines). The "Prerequisite" and "Update rules" sections, however, may need some adjustments to make them clearer.

Thanks for reading

[–] [email protected] 3 points 2 months ago (4 children)

Thank you for the feedback. I had a lot of fun playing with the model (and still have some improvements on my mind that might require porting it outside of Shadertoy)

Is there any part that was especially hard to understand ? I'm trying to make it as clear as possible for developers without a scientific background.

 

cross-posted from: https://lemmy.pierre-couy.fr/post/678825

Hi ! I've been working on this article for the past few days. It would mean a lot to me if you could provide some feedback.

It is about implementing a physico-chemical simulation as my first attempt to write a shader. The code is surprisingly simple and short (less than 100 lines). The "Prerequisite" and "Update rules" sections, however, may need some adjustments to make them clearer.

Thanks for reading

 

cross-posted from: https://lemmy.pierre-couy.fr/post/678825

Hi ! I've been working on this article for the past few days. It would mean a lot to me if you could provide some feedback.

It is about implementing a physico-chemical simulation as my first attempt to write a shader. The code is surprisingly simple and short (less than 100 lines). The "Prerequisite" and "Update rules" sections, however, may need some adjustments to make them clearer.

Thanks for reading

 

Hi ! I've been working on this article for the past few days. It would mean a lot to me if you could provide some feedback.

It is about implementing a physico-chemical simulation as my first attempt to write a shader. The code is surprisingly simple and short (less than 100 lines). The "Prerequisite" and "Update rules" sections, however, may need some adjustments to make them clearer.

Thanks for reading

[–] [email protected] 2 points 2 months ago

I can recommend some stuff I've been using myself :

  • Dolibarr as an ERP + CRM : requires some work to configure initially. As most (if not all) features are disabled by default, it requires enabling them based on what you need. It also has a marketplace with a bunch of modules you can buy
  • Gitea to manage codebases for customer projects. It can also do CI but I've not looked into it yet
  • Prometheus and its ecosystem (mostly promtail and grafana) for monitoring and alerting
  • docker mail server : makes it quite easy to self host a full mail server. The guides in their doc made it painless for me to configure dmarc/SPF/other stuff that make e-mail notoriously hard to host
  • Cal.com as a self hostable alternative to calendly
  • Authentik for single sign-on and centralized permission management
  • plausible for lightweight analytics
  • a mix of wireguard, iptables and nginx to basically achieve the same as cloudflare proxying and tunnels

I design, deploy and maintain such infrastructures for my own customers, so feel free to DM me with more details about your business if you need help with this

[–] [email protected] 11 points 2 months ago* (last edited 2 months ago) (2 children)

I did not read the link, but two of my biggest concerns do not appear in the summary you provided :

  • the burden of hosting an ActivityPub enabled service is often duplicated for each instance instead of being split between them (for example, my Lemmy instance has a large picture folder and database because it is replicating all posts from communities I'm subscribed to)
  • it's a privacy nightmare. All instance admins now have as much spying power as the single centralized service it is replacing

(Edit: typo)

[–] [email protected] 7 points 2 months ago (1 children)

It's a server that hosts map data for the whole world, and sends map fragments (tiles)as pictures for the coordinates and zoom levels that clients request from them

[–] [email protected] 5 points 2 months ago

Are you talking about Nginx Plus ? It seems to be a commercial product built on top of Nginx

[–] [email protected] 5 points 2 months ago (3 children)

According to the Wikipedia article, "Nginx is free and open-source software, released under the terms of the 2-clause BSD license"

Do you have any source about it going proprietary ?

[–] [email protected] 6 points 2 months ago (5 children)

It's still available in Debian's default repositories, so it must still be open source (at least the version that's packaged for Debian)

 

publication croisée depuis : https://lemmy.pierre-couy.fr/post/653426

This is a guide I wrote for Immich's documentation. It features some Immich specific parts, but should be quite easy to adapt to other use cases.

It is also possible (and not technically hard) to self-host a protomaps release, but this would require 100GB+ of disk space (which I can't spare right now). The main advantages of this guide over hosting a full tile server are :

  • it's a single nginx config file to deploy
  • it saves you some storage space since you're only hosting tiles you've previously viewed. You can also tweak the maximum cache size to your needs
  • it is easy to configure a trade-off between map freshness and privacy by tweaking the cache expiration delay

If you try to follow it, please send me some feedback on the content and the wording, so I can improve it

 

This is a guide I wrote for Immich's documentation. It features some Immich specific parts, but should be quite easy to adapt to other use cases.

It is also possible (and not technically hard) to self-host a protomaps release, but this would require 100GB+ of disk space (which I can't spare right now). The main advantages of this guide over hosting a full tile server are :

  • it's a single nginx config file to deploy
  • it saves you some storage space since you're only hosting tiles you've previously viewed. You can also tweak the maximum cache size to your needs
  • it is easy to configure a trade-off between map freshness and privacy by tweaking the cache expiration delay

If you try to follow it, please send me some feedback on the content and the wording, so I can improve it

 

publication croisée depuis : https://lemmy.pierre-couy.fr/post/584644

While monitoring my Pi-Hole logs today, I noticed a bunch of queries for XXXXXX.bodis.com, where XXXXXX are numbers. I saw a few variations for the numbers, each one being queried several times.

Digging further, I found out these queries were caused by CNAME records on domains that look like they used to point to Lemmy/Kbin instances.

From what I understand, domain owners can register a CNAME record to XXXXXX.bodis.com and earn some money from the traffic it receives. I guess that each number variation is a domain owner ID in Bodis' database. I saw between 5 to 10 different number variations, each one being pointed to by a bunch of old Lemmy domains.

This probably means that among actors who snatch expired domains, several of them have taken a specific interest with expired domains of old Lemmy instances. Another hypothesis is that there were a lot of domains registered for hosting Lemmy during the Reddit API debacle (about 1 year ago), which started expiring recently.

Are there any other instance admins who noticed the same thing ? Is any of my two hypothesis more plausible than the other ? Should we worry about this trend ?

Anyway, I hope this at least serves as a reminder to not let our domains expire ;)

 

While monitoring my Pi-Hole logs today, I noticed a bunch of queries for XXXXXX.bodis.com, where XXXXXX are numbers. I saw a few variations for the numbers, each one being queried several times.

Digging further, I found out these queries were caused by CNAME records on domains that look like they used to point to Lemmy/Kbin instances.

From what I understand, domain owners can register a CNAME record to XXXXXX.bodis.com and earn some money from the traffic it receives. I guess that each number variation is a domain owner ID in Bodis' database. I saw between 5 to 10 different number variations, each one being pointed to by a bunch of old Lemmy domains.

This probably means that among actors who snatch expired domains, several of them have taken a specific interest with expired domains of old Lemmy instances. Another hypothesis is that there were a lot of domains registered for hosting Lemmy during the Reddit API debacle (about 1 year ago), which started expiring recently.

Are there any other instance admins who noticed the same thing ? Is any of my two hypothesis more plausible than the other ? Should we worry about this trend ?

Anyway, I hope this at least serves as a reminder to not let our domains expire ;)

 

Cross-posted from : https://lemmy.pierre-couy.fr/post/581642

Context : Immich default map tile provider (which gets sent a bunch of PII every time you use the map feature) is a company that I see no reason to trust. This is a follow-up to this post, with the ~~permanent~~ temporary fix I came up with. I will also summarize the general opinion from the comments, as well as some interesting piece of knowledge that commenters shared.

Hacky fix

This will use Nginx proxy module to build a caching proxy in front of Open Street Map's tileserver and to serve a custom style.json for the maps.

This works well for me, since I already proxy all my services behind a single Nginx instance. It is probably possible to achieve similar results with other reverse proxies, but this would obviously need to be adapted.

Caching proxy

Inside Nginx's http config block (usually in /etc/nginx/nginx.conf), create a cache zone (a directory that will hold cached responses from OSM) :

http {
     # You should not need to edit existing lines in the http block, only add the line below
    proxy_cache_path /var/cache/nginx/osm levels=1:2 keys_zone=osm:100m max_size=5g inactive=180d;
}

You may need to manually create the /var/cache/nginx/osm directory and set its owner to Nginx's user (typically www-data on Debian based distros).

Customize the max_size parameter to change the maximum amount of cached data you want to store on your server. The inactive parameter will cause Nginx to discard cached data that's not been accessed in this duration (180d ~ 6months).

Then, inside the server block that serves your Immich instance, create a new location block :

server {
    listen 443 ssl;
    server_name immich.your-domain.tld;

    # You should not need to change your existing config, only add the location block below

    location /map_proxy/ {
        proxy_pass https://tile.openstreetmap.org/;
        proxy_cache osm;
        proxy_cache_valid 180d;
        proxy_ignore_headers Cache-Control Expires;
        proxy_ssl_server_name on;
        proxy_ssl_name tile.openstreetmap.org;
        proxy_set_header Host tile.openstreetmap.org;
        proxy_set_header User-Agent "Nginx Caching Tile Proxy for self-hosters";
        proxy_set_header Cookie "";
        proxy_set_header Referer "";
    }
}

Reload Nginx (sudo systemctl reload nginx). Confirm this works by visiting https://immich.your-domain.tld/map_proxy/0/0/0.png, which should now return a world map PNG (the one from https://tile.openstreetmap.org/0/0/0.png )

This config ignores cache control headers from OSM and sets its own cache validity duration (proxy_cache_valid parameter). After the specified duration, the proxy will re-fetch the tiles. 6 months seem reasonable to me for the use case, and it can probably be set to a few years without it causing issues.

Besides being lighter on OSM's servers, the caching proxy will improve privacy by only requesting tiles from upstream when loaded for the first time. This config also strips cookies and referrer before forwarding the queries to OSM, as well as set a user agent for the proxy following OSM foundation's guidelines (according to these guidelines, you should add a contact information to this user agent)

This can probably be made to work on a different domain than the one serving your Immich instance, but this probably requires to add the appropriate headers for CORS.

Custom style.json

I came up with the following mapstyle :

{
  "version": 8,
  "name": "Immich Map",
  "sources": {
    "immich-map": {
      "type": "raster",
      "tileSize": 256,
      "tiles": [
        "https://immich.your-domain.tld/map_proxy/{z}/{x}/{y}.png"
      ]
    }
  },
  "sprite": "https://maputnik.github.io/osm-liberty/sprites/osm-liberty",
  "glyphs": "https://fonts.openmaptiles.org/{fontstack}/{range}.pbf",
  "layers": [
    {
      "id": "raster-tiles",
      "type": "raster",
      "source": "immich-map",
      "minzoom": 0,
      "maxzoom": 22
    }
  ],
  "id": "immich-map-dark"
}

Replace immich.your-domain.tld with your actual Immich domain, and remember the absolute path you save this at.

One last update to nginx's config

Since Immich currently does not provide a way to manually edit style.json, we need to serve it from http(s). Add one more location block below the previous one :

location /map_style.json {
    alias /srv/immich/mapstyle.json;
}

Replace the alias parameter with the location where you saved the json mapstyle. After reloading nginx, your json style will be available at https://immich.your-domain.tld/map_style.json

Configure Immich to use this

For this last part, follow steps 8, 9, 10 from this guide (use the link to map_style.json for both light and dark themes). After clearing the browser or app's cache, the map should now be loaded from your caching proxy. You can confirm this by tailing Nginx's logs while you zoom and move around the map in Immich

Summary of comments from previous post

Self-hosting a tile server is not realistic in most cases

People who have previously worked with maps seem to confirm that there are no tile server solution lightweight enough to be self hosted by hobbyists. There is maybe some hope with generating tiles on demand, but someone with deep knowledge of the file formats involved in the process should confirm this.

Some interesting links were shared, which seem to confirm this is not realistically self-hostable with the available software :

General sentiment about this issue

In all this part, I want to emphasize that while there seems to be a consensus, this is only based on the few comments from the previous post and may be biased by the fact that we're discussing it on a non-mainstream platform. If you disagree with anything below, please comment this post and explain your point of view.

  • Nobody declared that they had noticed the requests to a third-party server before
  • A non-negligible fraction of Immich users are interested in the privacy benefits over other solutions such as Google photos. These users do not like their self-hosted services to send requests to third-party servers without warning them first
  • The fix should consist of the following :
    • Clearly document the implications of enabling the map, and any feature that sends requests to third parties
    • Disable by default features that send requests to third parties (especially if it contains any form of geolocated data)
    • Provide a way to easily change the tile provider. A select menu with a few pre-configured style.json would be nice, along with a way to manually edit style.json (or at least some of its fields) directly from the Immich config page
view more: next ›