this post was submitted on 13 Nov 2023
1 points (100.0% liked)

Data Hoarder

0 readers
3 users here now

We are digital librarians. Among us are represented the various reasons to keep data -- legal requirements, competitive requirements, uncertainty of permanence of cloud services, distaste for transmitting your data externally (e.g. government or corporate espionage), cultural and familial archivists, internet collapse preppers, and people who do it themselves so they're sure it's done right. Everyone has their reasons for curating the data they have decided to keep (either forever or For A Damn Long Time (tm) ). Along the way we have sought out like-minded individuals to exchange strategies, war stories, and cautionary tales of failures.

founded 1 year ago
MODERATORS
 

Hello πŸ‘‹

I'm feeling overwhelmed by the amount of work I created for my-self, my files are a mess.

  • How much time you spend just sorting and cleaning all you images?
  • Do you have the same folder structure for all you storage locations? (hard drives, cloud,...)
  • How many files do you have?
top 15 comments
sorted by: hot top controversial new old
[–] [email protected] 1 points 1 year ago (1 children)

Are they your files? Just dump them by major event (like "trip to", "X's wedding") or period if nothing special happened (/2023/"08 Home Summer" - I always put numerical month before so they get sorted). Then use some kind of software to organize them further (automatically, by place/date/face/object recognition, etc.) https://www.reddit.com/r/DataHoarder/comments/17l7230/bleedingedge_selfhosted_photo_software_in_2023_im/

[–] [email protected] 1 points 1 year ago (4 children)

The problem I have with all the softwares listed on the link, is that they need to run on a server. I want it on my hard drive

My problem: what if there is a selfie of my sister at her marriage with my grandma. I want to easily find this picture by searching "My sister Marriage" and by searching "Grandma"

How would you go about doing that? or I'm asking for too much?

[–] [email protected] 1 points 1 year ago (1 children)

You're not asking too much, but you will need a server (or some way to run the software) to actually create some sort of organization, image/object/face recognition processing, etc. that is indexed and can be searched. Depending on what tasks are being run will depend on how much system resources you'll need or what specific hardware is required to process the files.

Now, does the technology exist there to do what you are looking for without needing to get a server yourself? Yes, it absolutely does. However a lot of it can be found within Media Asset Management systems (not free) that connect to services that do the actual processing (costs extra and really not free). Like, if you wanted to search through your images and find every image with a red coffee cup in it, you could.

This also goes without saying, but if you have a single HDD with all your photos on it, the last thing you should be doing is running high I/O tasks on that HDD if that is your only copy. You should ingest those files into whatever solution you want and process it there, leaving the original intact. Ideally, duplicate that HDD and work off the duplicated data.

[–] [email protected] 1 points 1 year ago

Here is the thing: I don't trust cloud services to still exist in 40 years

Even open source ones, they are very complex and if the people maintaining it stop working on it, I'm doomed.

Is why I don't want a server to run.

I have seen people using Adobe Lightroom to do sort of what I'm looking for. But too expensive.

As a DataHorder, by using those servers you are talking about, are you sure you will still have access to them in the future?

[–] [email protected] 1 points 1 year ago (1 children)

My problem: what if there is a selfie of my sister at her marriage with my grandma. I want to easily find this picture by searching "My sister Marriage" and by searching "Grandma"

If you remember the year then you can quickly go to that folder just using Windows Explorer, then find the "Marriage" subfolder.

[–] [email protected] 1 points 1 year ago

But I can find a centralised place to see all the pictures of my grandma

[–] [email protected] 1 points 1 year ago

Might i suggest DigiKam ? Its free and open source, you can create albums and assign tags to pictures(like you said wedding 2006 or grandma), there is a tool for facial recognition but i dont have any experience with that. It creates few databases for the albums and i have no idea of it will handle that large amount of pictures but you can give it a try

[–] [email protected] 1 points 1 year ago

I organize my digital photos by date and time. I rename them using the embedded timestamp. There are various tools that can quickly rename all your photos with a timestamp as prefix. ExifTool, for example.

Once the photos are renamed with a timestamp prefixed, it is very easy to group and organize them. One folder per year. Subfolders per month or for special events. Add name of the event, persons and places to the filenames or subfolders.

[–] [email protected] 1 points 1 year ago

How do you organize 18,500 images on a hard drive?

Folder name: YYYY.MM.DD [Event Name] AB

The main copy is in my main external SSD. 1 copy backup in the backup external HDD, 1 Zipped backup in OneDrive.

Due to slow internet back then and in the case of the FAT32 system, I split every folder to 3,XX GB. So the "AB" is 01 02 03 for a large event. Like my wedding with >200 GB of photos and videos.

[–] [email protected] 1 points 1 year ago (1 children)

I put mine on a Synology station. The higher-end ones can run their photo organisation software, that auto-recognizes faces (mostly). This lets you sort by date, face, and some rudimentary categories (pet, landscape, food etc.). Also has phone apps so you can see your photos there, which is neat.

[–] [email protected] 1 points 1 year ago

I assume you're talking about the built-in Synology Photos app? I have a Synology NAS and keep putting off the decision for how I'm going to handle photo hosting. Ideally I'd want:

  • a unified archive (old stuff on random hard drives from both me and my wife, plus exporting/linking each of our icloud photo libraries and google photo libraries)
  • some kind of structured browsing, organizing things by timestamp (and including that in the filename) is fine
  • some kind of facial recognition, ideally a semi-manual approach that does its best on its own and then prompts for or accepts manual corrections
  • optionally, some kind of semi-manual de-duplication for near-identical images
  • optionally, an integrated way to share content, e.g. so I can text a friend a link to a specific album behind some kind of user authentication system
[–] [email protected] 1 points 1 year ago

similar situation a while back. I spent a couple month off an on, tagging all the photos to easily find in either shotwell or digikam. they're all in the same folder and new photos get tags as they get added. I took the tag approach because you can add pretty much an infinite number of tags to each photo. did that instead of trying to figure out where photos with multiple people, etc.. should go in an organized folder structure without having duplicates

[–] [email protected] 1 points 1 year ago

Definitely DigiKam! It's really fast (on Linux) and it has some powerful tools for organization. It also comes with face recognition and an AI that classifies your images based on quality. Everything is done on your computer.

[–] [email protected] 1 points 1 year ago