I hosted ad-free invision and phpbb forums for almost a decade before the centralized services started. I do it because I like social media and understand the value of a vibrant commons. I’ve been working with computers since I was a child so hosting yet-another-app-server is not really that big of a deal and gives my users and myself a place we can call home on the internet.
The out-of-pocket cost is minimal for a small instance, for larger instances scaling issues are going to hit eventually and costs will go beyond what 1-person will bear without complaint.
Highly recommend more smaller instances.
My server costs me only about $5 USD a month and I can host thousands of users without much additional effort (my sites before would usually run to ~10k users).
**Tl;dr we do this because we want to, the act itself is often fulfillment. **
I started !programming.dev because I am a moderator of several 100k+ subs over on Reddit and I didn’t want my communities to not have a place to go if Reddit crashed and burned (even though it’s incredibly unlikely). The main sub I moderated (/r/ExperiencedDevs) for years wanted user verification to combat the spam that was newbies commenting and posting about things they didn’t really know or understand. This will be possible to actually implement on Lemmy, whereas reddit was closed source, and didn’t really care about their communities.
I am also a strong supporter of pulling control away from megacorps. We need more small to medium sized businesses on the planet.
For selfish reasons? I wanted to work on something new and have true ownership over it, the ability to build a community that worked together to build something without capitalism standing in the way. It might seem strange, but one of the first things I did was bring multiple other people on board to help me maintain the server, even going so far as to add domain managers to the domain name. This was all to counter the major questions people were asking around “what if the host decides they don’t want to host anymore?”. Well hopefully the programming.dev community is willing to take that burden if the time ever comes, even though I hope it doesn’t. I also wanted to start something similar to a coop, where ownership is shared, meaning users have incentives to make the platform better. I have lots of ideas around this, but this will never be possible on Reddit. It is quite feasible here.
I also had the chance to buy an incredibly dope domain name! https://programming.dev! Why wouldn’t I jump at that chance? And I get to even use it instead of let it flounder. So many reasons to host something like this, to build a trusting community, a safe space to have to let people talk about a shared love/topic/hobby.
This will be possible to actually implement on Lemmy, whereas reddit was closed source, and didn’t really care about their communities.
Just curious how you think you might go about this. Do you plan on contributing yourself, forking, or using the community to influence the direction/prioritization of new features?
Honestly the domain name is a fantastic choice; Much easier to feel comfortable sharing links to my colleagues from programming.dev than something like “shit just works” or “lemmy world”
i just installed debian in an old laptop and paid google for a domain for $1 a month ($12/year).
doing it primarily because it seems like a cool thing to mess with and i’ve already learned cool things like DDNS and how to establish free https certificates via “let’s encrypt”. it’s pretty enlightening.
Does Google do some kind of reverse proxy, or are you not worried about your IP being fairly public? I’m interested in doing something similar and hosting at home, but am concerned about having my IP be so public. Obviously lemmy.world knows it, but that still seems better then any random troll knowing it.
I host my own to act as the sister of my Mastodon instance. It’s hard to afford given I’m a student, but it pays off knowing I’m on my little node of the decentralized internet.
I decided to host my own because I was on lemmy.world and we got blocked by beehaw, which had many of the communities I wanted to be a part of. I run mine on a server that’s in my house, so the only thing I’m paying for is electricity. And I have solar panels.
lol I was the same way. I went from not being able to see just beehaw to spotty federation with many other instances (oddly enough beehaw and the kbin instances are nearly perfect), so I’m not yet sure if it was an upgrade.
What’s funny is I feel like my instance has spotty federation with lemmy.world, but everything else is pretty good. I wonder what’s going on over there.
It was really annoying to set up if I’m being honest. If I hadn’t taken classes on Docker, I would have never figured it out. Luckily they have been improving the process recently. It already much easier now than it was a week ago. Hopefully by the next major release it is easy peasy.
It’s an amazing software. Just today I needed to spin up a super simple webserver just to share a file with another program who needed to download it via https. Literally one command and I had an Apache server serving an arbitrary folder with a specified SSL certificate for HTTPS. And then one more command to shut it down, and it’s gone. No software to uninstall when I don’t need it anymore. No leftover files (well, the images are cached, but that’s super easy to clear).
And then just a few hours later, I wanted to test the same program using a samba share instead of https. One command, bam. Sharing the same folder with samba. And then one command and it’s gone.
And say you want to upgrade your apache version later, you don’t need to worry about if your package manager is pointing to the newest version. You just restart your container with the latest tag and you have the latest version.
I’m actually rewriting a lot of my services to be in containers. I host a few discord bots for a community that my wife is an admin of. I accidentally updated my server’s version of python once and nuked a few bots (Discord’s API updated and I had to change some code in the new version). I containerized my bots, and now they will always have their own python version independent of my server’s version. And I made sure to specify a version tag rather than using the latest tag, so they will never change. It makes it way easier to make sure it never breaks.
I don’t host a lemmy instance yet, but I host other services and I do it mainly for fun. As far as I have read, a small scale lemmy instance runs very well on very low performance hardware, which means the cost of running it is probably less than 10 $ a month.
The big lemmy instances that need better hardware have patreons and other services where people can donate money.
I asked another commenter here the same question that I’m going to ask you.
I was running a my own single user instance, but I had a hard time getting comments to come in to mine. Have you found a good way to get Lemmy to more reliably pull in comments from remote instances?
I would have to browse out to the original instance to see most of the comments, then back to my own if I wanted to comment (if it was under a comment that my instance hadn’t pulled in, then too bad), then back out to the main site to continue reading. I found that process very tedious so I switch back to a more populated server which seems to pull in most if not all of the comments.
I have not noticed any issues like you describe. Everything seems to dtrt. To be fair, it could be happening and I don’t know it, but I’m not going to go out of my way to look for comments that clearly don’t like me, either.
It’s worth spending the time figuring out, I think. They do pro rated pricing, so play with it and make mistakes, it won’t cost you much to create and destroy some servers if you start from the cheapest to play around with.
I chose to host my own instance for my community because I wanted to be independent from any other instances’ administration, federation decisions or any sorts of politics.
Right now I’m paying for it out of my own pocket, but I’m working towards setting up a donation flow.
I host my own instance for the same reason I self-host the dozens of other services. To have control over my digital services the best I can. I have a few server machines running various services. I run like 40 docker containers running at the moment. Lemmy is a set of those containers.
It cost me the electricity cost to run the server, plus the cost of my internet. I suppose you should include the initial hardware cost- my servers are basically my old gaming rigs. Not to mention the time investment to maintain another service. For me, it’s worth it to self-host if I can.
Specifically for lemmy, I seen how overloaded the various major lemmy instances out there were, so self hosting could mean one less user on those instances. I also didn’t see any significant drawbacks to self-hosting the instance since I can still join and communicate with all the other communities.
I tried out my own, one person instance, and I had a hard time getting comments to pull in. I would have to browse to the original instance to view all of the comments. Have you found a good way to overcome that?
I switched back to a more public instance just because I found the process of going out to view the content, back to my own instance if I wanted to comment, then back to the original again to keep reading the discussion very tedious.
Being on a more populated server seems to give most if not all of the comments directly.
When you first subscribe to a community it only pulls in the last 20 or so posts and I think a limited amount of comments, and then everything going forward. This seems to be a common point of confusion for a lot of instance admins.
Presumably this is to prevent a possible DDoS/performance failure vector as it would be trivial to setup a large swarm of instances on tiny VM’s and then simultaneously start hitting massive communities from a single instance and requesting a large body of historical content.
Edit: Also when you first setup and start subscribing to a large number of instances, this initiates a LOT of communication and database writes. Lemmy still has some performance bottlenecks. Once everything is initially synced and settled it runs fine. I have a friend running their instance on a $5 Linode instance that only has 1vCPU and 1GB of RAM without any issues, and they’re hosting users.
I ran mine for a couple weeks, and communities I’d been subscribed to from day 1 were still missing most comments on the posts unless I clicked through to their page. Maybe there was something funky with my install, but I used Lemmy’s ansible scripts to deploy so I don’t know what else I could do.
I really like the overall concept of Lemmy, so I decided to set up lemm.ee to support the Lemmy network with my skillset. I have previously had the privilege of being responsible for running large platforms online (end-to-end, everything from operations to software engineering), and so far, this experience seems to be extremely relevant for running Lemmy in its current state.
As for paying for hosting, my initial plan was to to just pay for everything myself as kind of a hobby, but the userbase at lemm.ee has been very gracious in first asking me several times to share costs, and then actually sending money once I set up donations. I’m not sure yet if this donations-based funding will be sustainable, or if it will fall off after the initial hype dies, but for now it’s really awesome to see that there are several other people who believe in lemm.ee and want to share financial responsibility for it.
What does it actually cost? I have no basis for a ballpark guess even. I’ve seen this question asked to a number of admins and haven’t seen a direct answer.
It’s hard to judge how sustainable a donation based approach is without that info.
The costs will vary wildly depending on how the instance has been set up. If you set up all necessary services on a single VPS (as is the most common approach for smaller instances), then you can probably get by on $10-$20 a month. Splitting different services onto different servers, adding backups, load balancing, CDNs, redundancy, caches, etc will quickly increase the cost. Bigger instances need more powerful servers, that will increase the cost further.
On lemm.ee, we are currently not using very high-end servers, but we ARE using all the other things I mentioned above, and the monthly cost is currently hovering around $200 (that’s for 3 servers, a managed database, object storage, load balancing, a global CDN, and an e-mail provider). This is still on the very cheap side in the grand scheme of running online platforms, but definitely much more than I would want to pay for a single-user instance for example.
$200 is still on the low end, trust me - high end managed databases and compute resources are in the thousands, adding redundancy to that will double or triple it 😃
Ye if you’re adding like super redundancies etc sure. But I mean, it’s lemmy. We don’t need all that. My whole VM config is in ansible. I can literally scrap the whole thing, and redeploy it in 10 minutes. I just need to have a DB backup in case of some sort of catastrophic failure
IaC is awesome 👍 lemm.ee is deployed using Terraform.
My lemmy_server redundancy is mostly just so I can do infra changes without downtime (just take one node out of the load balancer, redeploy it and put the new one in), but it does also help a bit with general performance. I agree it’s not strictly necessary, the vast majority of other instances are just running on a single server, but I do think it’s very nice to have.
For back-ups, I have point-in-time recovery, so I can restore the database to any random timestamp - and it has actually come in useful once already when two weeks ago I was able to restore to a good state about a few minutes after a problem - I think nobody even noticed that anything happened in that case 😃
Problem with Terraform is it assumes cloud providers, which tend to be expensive (or at least, I don’t know a way to do terraform on hetzner dedis :D )
My solution is to get cheapskate VPS and dedis and loadbalance them as frontends. The VM request is manual, but I only have to do this once anyway. It’s what I’m currently doing with the AI Horde. Of course, that doesn’t help when there’s DB changes but still.
What object storage are you using btw? I’m thinking to move to R2 or smt since I’ve had good experience with them until now. Contabo’s is way cheaper but when I tried to use it for high-demand stuff it dropped dead on the spot. But it might be ok for Lemmy.
I can’t thank you enough for sharing your knowledge, I am very interested in learning about server management and being able to read your thoughts is something I find extremely invaluable, please keep up with the great work! 😁
Yeah pretty much my setup. Full DB and pictrs backups uploaded to Backblaze B2 every few hours. Config files all backed up. Also send a copy to my home lab. Encrypted on the server before it gets sent out.
Worst case - we lose a few hours of data.
I keep about a week’s worth of backups on B2 (and 2 days on my home server) which is just over 200GB now. But that costs next to nothing on B2.
Maybe eventually if my instance grows more I might consider doing a replica DB on another server.
My total costs are like, $7.50 a month but I only have 15 actually active users. I don’t need to grow, but I’m willing to. If the costs increase too much I’ll ask for donations and if that stops covering things I’ll just close registrations. That’s the great part of not being a business chasing infinite growth.
That’s great! It’s kind of a crowdfunded instance, then. Makes me wonder if it would be feasible to implement some sort of collection box plugin or something…
Yes, and I know it’s counter to the core motivations of this movement, but probably need a centralized repository for donation that can be a universal door for funds that can then be distributed to vulnerable, but active, instances. Needs to be run by a collective of reps from instances meeting a minimum threshold of support for the community. Also needs to be nimble enough to revoke funding is an instance takes a hard evil turn.
Or maybe just an app/site that recommends a distribution of a set monthly amount (e.g. 30 bucks) to the instances you use the most as a user?
I don’t think we need that at all. Each instance can look after its own hosting and funding.
A centralised pot is asking for trouble - what about theft or corruption? What happens about disagreements like defederation? Seems a lot more trouble than it’s worth.
Yes … if anyone is a developer looking for ways to provide value to the fediverse … I suspect the donation process is probably of high value.
I don’t know the best way for it to be done … but something so that it’s easy for users to setup a single or regular donation and easy for devs and admins to put the relevant button right into their platform … all so that whoever is willing to donate has every opportunity to do so.
I’ve been hosting a gaming server plus other related stuff myself for some years now.
While the user base will definitely be different, relying solely on donations is unfortunately not sustainable long-term. Donations fluctuate massively based on time of year in my experience.
So it’s always good to periodically remind your community that lemm.ee needs donations to survive long-term.
When I do those reminders, users come out of the woods in droves to donate. It’s less that they’re unwilling to donate and more that they just forget to donate.
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !asklemmy@lemmy.ml
A loosely moderated place to ask open-ended questions
If your post meets the following criteria, it’s welcome here!
Open-ended question
Not offensive: at this point, we do not have the bandwidth to moderate overtly political discussions. Assume best intent and be excellent to each other.
Not regarding using or support for Lemmy: context, see the list of support communities and tools for finding communities below
Not ad nauseam inducing: please make sure it is a question that would be new to most members
I hosted ad-free invision and phpbb forums for almost a decade before the centralized services started. I do it because I like social media and understand the value of a vibrant commons. I’ve been working with computers since I was a child so hosting yet-another-app-server is not really that big of a deal and gives my users and myself a place we can call home on the internet.
The out-of-pocket cost is minimal for a small instance, for larger instances scaling issues are going to hit eventually and costs will go beyond what 1-person will bear without complaint.
Highly recommend more smaller instances.
My server costs me only about $5 USD a month and I can host thousands of users without much additional effort (my sites before would usually run to ~10k users).
**Tl;dr we do this because we want to, the act itself is often fulfillment. **
Mostly to test about how activitypub works, also paying out of pocket for $5 a month.
I started !programming.dev because I am a moderator of several 100k+ subs over on Reddit and I didn’t want my communities to not have a place to go if Reddit crashed and burned (even though it’s incredibly unlikely). The main sub I moderated (/r/ExperiencedDevs) for years wanted user verification to combat the spam that was newbies commenting and posting about things they didn’t really know or understand. This will be possible to actually implement on Lemmy, whereas reddit was closed source, and didn’t really care about their communities.
I am also a strong supporter of pulling control away from megacorps. We need more small to medium sized businesses on the planet.
For selfish reasons? I wanted to work on something new and have true ownership over it, the ability to build a community that worked together to build something without capitalism standing in the way. It might seem strange, but one of the first things I did was bring multiple other people on board to help me maintain the server, even going so far as to add domain managers to the domain name. This was all to counter the major questions people were asking around “what if the host decides they don’t want to host anymore?”. Well hopefully the programming.dev community is willing to take that burden if the time ever comes, even though I hope it doesn’t. I also wanted to start something similar to a coop, where ownership is shared, meaning users have incentives to make the platform better. I have lots of ideas around this, but this will never be possible on Reddit. It is quite feasible here.
I also had the chance to buy an incredibly dope domain name! https://programming.dev! Why wouldn’t I jump at that chance? And I get to even use it instead of let it flounder. So many reasons to host something like this, to build a trusting community, a safe space to have to let people talk about a shared love/topic/hobby.
Just curious how you think you might go about this. Do you plan on contributing yourself, forking, or using the community to influence the direction/prioritization of new features?
Honestly the domain name is a fantastic choice; Much easier to feel comfortable sharing links to my colleagues from programming.dev than something like “shit just works” or “lemmy world”
Totally agree with the dope domain name. Not going to lie, a big reason for picking
programming.devwas to be /u/jim@programming.devi just installed debian in an old laptop and paid google for a domain for $1 a month ($12/year).
doing it primarily because it seems like a cool thing to mess with and i’ve already learned cool things like DDNS and how to establish free https certificates via “let’s encrypt”. it’s pretty enlightening.
Does Google do some kind of reverse proxy, or are you not worried about your IP being fairly public? I’m interested in doing something similar and hosting at home, but am concerned about having my IP be so public. Obviously lemmy.world knows it, but that still seems better then any random troll knowing it.
I host my own to act as the sister of my Mastodon instance. It’s hard to afford given I’m a student, but it pays off knowing I’m on my little node of the decentralized internet.
I decided to host my own because I was on lemmy.world and we got blocked by beehaw, which had many of the communities I wanted to be a part of. I run mine on a server that’s in my house, so the only thing I’m paying for is electricity. And I have solar panels.
lol I was the same way. I went from not being able to see just beehaw to spotty federation with many other instances (oddly enough beehaw and the kbin instances are nearly perfect), so I’m not yet sure if it was an upgrade.
What’s funny is I feel like my instance has spotty federation with lemmy.world, but everything else is pretty good. I wonder what’s going on over there.
Clever
How difficult is it to host your own instance? Can you still use apps like Connect with it?
It was really annoying to set up if I’m being honest. If I hadn’t taken classes on Docker, I would have never figured it out. Luckily they have been improving the process recently. It already much easier now than it was a week ago. Hopefully by the next major release it is easy peasy.
Good to know! Sounds like another reason for me to start learning Docker.
It’s an amazing software. Just today I needed to spin up a super simple webserver just to share a file with another program who needed to download it via https. Literally one command and I had an Apache server serving an arbitrary folder with a specified SSL certificate for HTTPS. And then one more command to shut it down, and it’s gone. No software to uninstall when I don’t need it anymore. No leftover files (well, the images are cached, but that’s super easy to clear).
And then just a few hours later, I wanted to test the same program using a samba share instead of https. One command, bam. Sharing the same folder with samba. And then one command and it’s gone.
And say you want to upgrade your apache version later, you don’t need to worry about if your package manager is pointing to the newest version. You just restart your container with the latest tag and you have the latest version.
I’m actually rewriting a lot of my services to be in containers. I host a few discord bots for a community that my wife is an admin of. I accidentally updated my server’s version of python once and nuked a few bots (Discord’s API updated and I had to change some code in the new version). I containerized my bots, and now they will always have their own python version independent of my server’s version. And I made sure to specify a version tag rather than using the latest tag, so they will never change. It makes it way easier to make sure it never breaks.
Yes to the last question
As long as you have a domain or IP address for the instance, the apps should be able to connect to it.
Run it on some free or cheap cloud instance https://paul.totterman.name/posts/free-clouds/
I don’t host a lemmy instance yet, but I host other services and I do it mainly for fun. As far as I have read, a small scale lemmy instance runs very well on very low performance hardware, which means the cost of running it is probably less than 10 $ a month.
The big lemmy instances that need better hardware have patreons and other services where people can donate money.
I saw the struggle on the main instance, and wanted to help.
I pay $12/month for a dedicated vps.
I asked another commenter here the same question that I’m going to ask you.
I was running a my own single user instance, but I had a hard time getting comments to come in to mine. Have you found a good way to get Lemmy to more reliably pull in comments from remote instances?
I would have to browse out to the original instance to see most of the comments, then back to my own if I wanted to comment (if it was under a comment that my instance hadn’t pulled in, then too bad), then back out to the main site to continue reading. I found that process very tedious so I switch back to a more populated server which seems to pull in most if not all of the comments.
I have not noticed any issues like you describe. Everything seems to dtrt. To be fair, it could be happening and I don’t know it, but I’m not going to go out of my way to look for comments that clearly don’t like me, either.
It was very obvious for me. A post would only have one comment, but then dozens when I clicked to the original instance.
Maybe I’ll try to spin up a brand new one and see if it’s any better.
Ah, tbh I wouldn’t have ever checked other instances for something like that. But I’m lazy, ha
That’s a great price for a vps. What host do you use?
Digital Ocean. It’s just a small 2gig server, but it’s doing the job like a champ so far.
that’s really cool thanks
Maybe I should figure out DO. I disregarded then because I didn’t understand their droplets or whatever they’re called.
It’s worth spending the time figuring out, I think. They do pro rated pricing, so play with it and make mistakes, it won’t cost you much to create and destroy some servers if you start from the cheapest to play around with.
They seem more node.js-friendly than other hosts I’ve tried. I might have to take the plunge.
I chose to host my own instance for my community because I wanted to be independent from any other instances’ administration, federation decisions or any sorts of politics.
Right now I’m paying for it out of my own pocket, but I’m working towards setting up a donation flow.
like many others, I hosted my own for a variety of reasons.
As for the cost, I have several VPS’s which cost ~$40/year, so the cost is basically negligible.
Can I ask who your VPS provider is?
$3.33/month? I too would like to know who your VPS provider is.
Not the poster above or if I understand the difference between a VPS and an unmanaged VM, but I currently have 2 VMs running:
I’ve been tinkering and selfhosting a few applications and a subsonic music server. Works great!
I host my own instance for the same reason I self-host the dozens of other services. To have control over my digital services the best I can. I have a few server machines running various services. I run like 40 docker containers running at the moment. Lemmy is a set of those containers.
It cost me the electricity cost to run the server, plus the cost of my internet. I suppose you should include the initial hardware cost- my servers are basically my old gaming rigs. Not to mention the time investment to maintain another service. For me, it’s worth it to self-host if I can.
Specifically for lemmy, I seen how overloaded the various major lemmy instances out there were, so self hosting could mean one less user on those instances. I also didn’t see any significant drawbacks to self-hosting the instance since I can still join and communicate with all the other communities.
I tried out my own, one person instance, and I had a hard time getting comments to pull in. I would have to browse to the original instance to view all of the comments. Have you found a good way to overcome that?
I switched back to a more public instance just because I found the process of going out to view the content, back to my own instance if I wanted to comment, then back to the original again to keep reading the discussion very tedious.
Being on a more populated server seems to give most if not all of the comments directly.
When you first subscribe to a community it only pulls in the last 20 or so posts and I think a limited amount of comments, and then everything going forward. This seems to be a common point of confusion for a lot of instance admins.
Presumably this is to prevent a possible DDoS/performance failure vector as it would be trivial to setup a large swarm of instances on tiny VM’s and then simultaneously start hitting massive communities from a single instance and requesting a large body of historical content.
Edit: Also when you first setup and start subscribing to a large number of instances, this initiates a LOT of communication and database writes. Lemmy still has some performance bottlenecks. Once everything is initially synced and settled it runs fine. I have a friend running their instance on a $5 Linode instance that only has 1vCPU and 1GB of RAM without any issues, and they’re hosting users.
I ran mine for a couple weeks, and communities I’d been subscribed to from day 1 were still missing most comments on the posts unless I clicked through to their page. Maybe there was something funky with my install, but I used Lemmy’s ansible scripts to deploy so I don’t know what else I could do.
I can see that, but I expect the main cost of servers are the hosting of subs, not maintaining users.
I expect that those servers are going to need support eventually.
Which is easy to avoid, if you want. You can disallow the creation of communities on your instance.
So users can still join and make it their home, but they will have to subscribe to communities on other instances.
I’m not sure about the ethical implications. It means you’re sort of leeching off moderation and maintenance from other admins.
I guess it’s a negligible point and totally fine, but not sure. Would be interesting if community-admins could share their thoughts on this.
But if everyone avoids hosting communities, then Lemmy is going to die because no one is going to be in a community.
It may not be an ethical issue, but it is a financial one that needs to be addressed.
I host my own, and I already have servers for other reasons so there’s effectively no extra cost because I can easily handle the load.
I’ve just launched one:
lemmy.fosshost.com
Why?
I really like the overall concept of Lemmy, so I decided to set up lemm.ee to support the Lemmy network with my skillset. I have previously had the privilege of being responsible for running large platforms online (end-to-end, everything from operations to software engineering), and so far, this experience seems to be extremely relevant for running Lemmy in its current state.
As for paying for hosting, my initial plan was to to just pay for everything myself as kind of a hobby, but the userbase at lemm.ee has been very gracious in first asking me several times to share costs, and then actually sending money once I set up donations. I’m not sure yet if this donations-based funding will be sustainable, or if it will fall off after the initial hype dies, but for now it’s really awesome to see that there are several other people who believe in lemm.ee and want to share financial responsibility for it.
What does it actually cost? I have no basis for a ballpark guess even. I’ve seen this question asked to a number of admins and haven’t seen a direct answer.
It’s hard to judge how sustainable a donation based approach is without that info.
The costs will vary wildly depending on how the instance has been set up. If you set up all necessary services on a single VPS (as is the most common approach for smaller instances), then you can probably get by on $10-$20 a month. Splitting different services onto different servers, adding backups, load balancing, CDNs, redundancy, caches, etc will quickly increase the cost. Bigger instances need more powerful servers, that will increase the cost further.
On lemm.ee, we are currently not using very high-end servers, but we ARE using all the other things I mentioned above, and the monthly cost is currently hovering around $200 (that’s for 3 servers, a managed database, object storage, load balancing, a global CDN, and an e-mail provider). This is still on the very cheap side in the grand scheme of running online platforms, but definitely much more than I would want to pay for a single-user instance for example.
And honestly, 200 is on the high-end even with this setup. lemmy.dbzer0.com is way less
$200 is still on the low end, trust me - high end managed databases and compute resources are in the thousands, adding redundancy to that will double or triple it 😃
Ye if you’re adding like super redundancies etc sure. But I mean, it’s lemmy. We don’t need all that. My whole VM config is in ansible. I can literally scrap the whole thing, and redeploy it in 10 minutes. I just need to have a DB backup in case of some sort of catastrophic failure
IaC is awesome 👍 lemm.ee is deployed using Terraform.
My
lemmy_serverredundancy is mostly just so I can do infra changes without downtime (just take one node out of the load balancer, redeploy it and put the new one in), but it does also help a bit with general performance. I agree it’s not strictly necessary, the vast majority of other instances are just running on a single server, but I do think it’s very nice to have.For back-ups, I have point-in-time recovery, so I can restore the database to any random timestamp - and it has actually come in useful once already when two weeks ago I was able to restore to a good state about a few minutes after a problem - I think nobody even noticed that anything happened in that case 😃
Problem with Terraform is it assumes cloud providers, which tend to be expensive (or at least, I don’t know a way to do terraform on hetzner dedis :D )
My solution is to get cheapskate VPS and dedis and loadbalance them as frontends. The VM request is manual, but I only have to do this once anyway. It’s what I’m currently doing with the AI Horde. Of course, that doesn’t help when there’s DB changes but still.
What object storage are you using btw? I’m thinking to move to R2 or smt since I’ve had good experience with them until now. Contabo’s is way cheaper but when I tried to use it for high-demand stuff it dropped dead on the spot. But it might be ok for Lemmy.
I can’t thank you enough for sharing your knowledge, I am very interested in learning about server management and being able to read your thoughts is something I find extremely invaluable, please keep up with the great work! 😁
Yeah pretty much my setup. Full DB and pictrs backups uploaded to Backblaze B2 every few hours. Config files all backed up. Also send a copy to my home lab. Encrypted on the server before it gets sent out.
Worst case - we lose a few hours of data.
I keep about a week’s worth of backups on B2 (and 2 days on my home server) which is just over 200GB now. But that costs next to nothing on B2.
Maybe eventually if my instance grows more I might consider doing a replica DB on another server.
My total costs are like, $7.50 a month but I only have 15 actually active users. I don’t need to grow, but I’m willing to. If the costs increase too much I’ll ask for donations and if that stops covering things I’ll just close registrations. That’s the great part of not being a business chasing infinite growth.
That’s great! It’s kind of a crowdfunded instance, then. Makes me wonder if it would be feasible to implement some sort of collection box plugin or something…
Yes, and I know it’s counter to the core motivations of this movement, but probably need a centralized repository for donation that can be a universal door for funds that can then be distributed to vulnerable, but active, instances. Needs to be run by a collective of reps from instances meeting a minimum threshold of support for the community. Also needs to be nimble enough to revoke funding is an instance takes a hard evil turn.
Or maybe just an app/site that recommends a distribution of a set monthly amount (e.g. 30 bucks) to the instances you use the most as a user?
I don’t think we need that at all. Each instance can look after its own hosting and funding.
A centralised pot is asking for trouble - what about theft or corruption? What happens about disagreements like defederation? Seems a lot more trouble than it’s worth.
Yes … if anyone is a developer looking for ways to provide value to the fediverse … I suspect the donation process is probably of high value.
I don’t know the best way for it to be done … but something so that it’s easy for users to setup a single or regular donation and easy for devs and admins to put the relevant button right into their platform … all so that whoever is willing to donate has every opportunity to do so.
I’ve been hosting a gaming server plus other related stuff myself for some years now.
While the user base will definitely be different, relying solely on donations is unfortunately not sustainable long-term. Donations fluctuate massively based on time of year in my experience. So it’s always good to periodically remind your community that lemm.ee needs donations to survive long-term.
When I do those reminders, users come out of the woods in droves to donate. It’s less that they’re unwilling to donate and more that they just forget to donate.