NeoJackOfBlades

joined 11 months ago
[–] [email protected] 1 points 10 months ago

Ubuntu server with ansible playbook and docker services

[–] [email protected] 1 points 10 months ago (1 children)

I'm sorry about that, is your Operating System Unix or Windows? x86 or arm? I tested it on Ubuntu server and fedora and it is correct, I will test it on Windows soon. I'm trying to imagine what it could still be.

[–] [email protected] 1 points 10 months ago

Hello, criticism is certainly welcome!
If you can open an issue on github it will be easier for me to follow, as I may not see these comments.
About the message date, you are right! Until I divulge the project, I was the only user so I didn't know it could have multiple types of message date formats, then I developed for a specific format (which isn't even uncommon).
Someone had already warned me about this and so I started to develop something that could format the date message in a rigth way. It's almost ready, but unfortunately, there are dates that
end up being ambiguous like 01/01/2023 and it's not possible to infer the correct format, so I'll probably have to create an environment variable for that, which I really didn't want.
If you look on github I opened a bug issue for this (although maybe it's not really a bug but rather an improvement, because it works but with a specific format) and in the Github's Projects part it's already under development.
Regarding the duplication of data, I mentioned it in a comment here on this post, but perhaps I should have made it clearer. Anyway, as I said in the post, the project is still far from ideal, despite it working very well for my use case (every import I do is always new messages).
Anyway, I know this is a very important point, so I created a way of deduplication considering the last message in the database as a cutoff parameter. This is already in the latest version of the docker image.
Regarding docker, more information about the error is welcome, you weren't the first person to talk about it but I couldn't replicate the problem. I tested it on Fedora and the Ubuntu server, I built it locally, I pulled it from the Registry, did docker system prune --volumes -a, and it still worked as expected.

[–] [email protected] 1 points 10 months ago (3 children)

I just cleared all images and containers to make sure I wasn't working with any cache and that everything went fine.

This message is expected because we are not defining any profiles: No active profile set, falling back to 1 default profile: "default".

Then I would have to see what error happened after that. The only properties required are the database connection properties.That being said, you can run compose.yml in the project root.It will build an image locally.Or replace the build: ./ line with the image: ghcr.io/vitormarcal/chatvault:latest

docker-compose -f compose.yml

I tested these two ways here and they continue to work. Have you added database information?

[–] [email protected] 1 points 10 months ago

For now, you enter the Whatsapp interface, go to export chat, choose whether you want to send it by email or on disk. With these files you can configure import via email, import through the interface with the zip file or place them in a specific folder, so that you can trigger the disk scan through the chatvault interface.

[–] [email protected] 1 points 10 months ago

For now, the application cannot deal with the physical base of WhatsApp, the application can import based on the txt of messages and media that WhatsApp generates when exported through the application interface

[–] [email protected] 1 points 10 months ago

I worked on a similar project for a long time...I don't really have code that I can share right now, but perhaps I can offer some hints.

Instead of exporting chats from the app (since the function is too limited) I decrypted the backup database that WhatsApp creates. This was originally possible only on rooted Android phones, but it's now possible if the user has chosen to encrypt the backup (and the encryption key is known).

Once the database is decrypted, it is just a matter of extracting messages and media information from it, and rebuild the chat. One of the problems here (and one of the reasons why I could not keep spending time on this project) is that Meta changed the structure of the database multiple times over the last years.

Here you can see an example of a chat I recreated from the database. The display format is HTML, trying to replicate the WhatsApp original look.

Let me know if you want more info. I will be happy to share!

EDIT: It seems I cannot post a picture in the comment. Is that a limitations of this community?

I didn't know you could access these messages without root these days, good to know!
I was already aware of these frequent structure changes, so I went with the simplest as a proof of concept.
Anyway, I plan to study a little more to find out how the Whatsapp base works, I don't rule out an experimental import of this type.
Thanks for the information! :D

 

I think this is a pretty niche demand and probably another topic for r/DataHoarder but anyway, here I am.

I created this application to basically have a way to store my WhatsApp messages away from the Google/Meta servers. Or at least not depend so much on Google backup.

Whatsapp has a very limited export functionality, which any user can use through the app's own interface. Once these messages and media have been exported, you can place them in a folder monitored by ChatVault, send them to an email monitored by ChatVault or upload them via the interface. Once ingested by chatvault, it will record the chat media on disk and save the messages in a database in a structured way. These messages can be accessed in a front end similar to a chat application.

It's still under development, some things need to be improved (mainly the UI), it's still far from ideal, it's true, the way Whatsapp allows us to export messages is quite bad, which makes the entire process of exporting and ingesting it into chatvault quite coupled but it can still be useful for those who want to store their messages independently, just like I wanted.

https://github.com/vitormarcal/chatvault

Edit: add an application interface image

The UI still needs some work, but it serves the purpose