Adventures in web community infrastructure
Moderator: TMAX
Re: Adventures in web community infrastructure
Ditto what Dusty said! Jack
- stuartcnz
- Site Admin
- Posts: 875
- Joined: Fri Dec 19, 2008 8:05 pm
- Location: Aotearoa, New Zealand
- Contact:
Re: Adventures in web community infrastructure
Among the many, one of the big challenges is going to be, to convert the existing database from MySQL, to PostgreSQL. The reason for that is that the server that I intend on putting the new forum on, already has a running PostgreSQL server. It doesn't make sense to try and also run a MySQL server on it as well. As far as I know, both the existing forum software, and what I want to convert to, are happy running on either of the two types of database. I'm not so sure that converting the database itself, is so straight forward though.
https://stuartcrawfordmedia.com/
https://nomadichome.org/
https://ethicallogistics.com/ Challenging the way you think.
JID:stuart@nomadicista.nz
https://nomadichome.org/
https://ethicallogistics.com/ Challenging the way you think.
JID:stuart@nomadicista.nz
- stuartcnz
- Site Admin
- Posts: 875
- Joined: Fri Dec 19, 2008 8:05 pm
- Location: Aotearoa, New Zealand
- Contact:
Re: Adventures in web community infrastructure
In order to figure out how to convert the MySQL database to PostgreSQL, I have set up a virtual machine on my desktop computer. I have installed the forum software and imported a recent snapshot of the forum database.
The first tool that I tried to do the conversion, is called pgloader. So far I have not been able to get it to work. My next trick will be to try another tool called FromMySqlToPostgreSql which, from what I can tell, is a PHP script, using a configuration file, customized for the particular database to be converted.
If I can get that to work, and can verify that it works with the forum software on the virtual machine, then it would be time to convert the forum database, to run on Postgres.
The first tool that I tried to do the conversion, is called pgloader. So far I have not been able to get it to work. My next trick will be to try another tool called FromMySqlToPostgreSql which, from what I can tell, is a PHP script, using a configuration file, customized for the particular database to be converted.
If I can get that to work, and can verify that it works with the forum software on the virtual machine, then it would be time to convert the forum database, to run on Postgres.
https://stuartcrawfordmedia.com/
https://nomadichome.org/
https://ethicallogistics.com/ Challenging the way you think.
JID:stuart@nomadicista.nz
https://nomadichome.org/
https://ethicallogistics.com/ Challenging the way you think.
JID:stuart@nomadicista.nz
- stuartcnz
- Site Admin
- Posts: 875
- Joined: Fri Dec 19, 2008 8:05 pm
- Location: Aotearoa, New Zealand
- Contact:
Re: Adventures in web community infrastructure
A bit of a side track, but part of the process: I wanted to be able to host my own photo's for posting on the forum, and possibly other places. I have just been sticking them in the back of other sites, but it was both a bit of a hassle to do, and a bit messy.
So I have been hunting around for something simple, and written in Python to do it, and preferably without needing to run a database for it.
Last night, I found an example that looked good, but couldn't access the site, that hosts the code. This morning I managed to get a copy through archive.org. Turns out the software was last updated about ten years ago. This meant that it didn't immediately work.
I managed to get it going, and decided to publicly host it on my git repository at https://kallithea.ethicallogistics.com/gallery2.py in case anyone else wants to use it.
It was a good exercise for me, as it is the first time I have been successful in fixing a python program by myself. Which is hopefully going to make it just that little bit easier to convert the forum in the future.
Here is the first gallery. Most of these photo's are already on the forum in different places. https://stuartcrawford.nz/pics/index.html
So I have been hunting around for something simple, and written in Python to do it, and preferably without needing to run a database for it.
Last night, I found an example that looked good, but couldn't access the site, that hosts the code. This morning I managed to get a copy through archive.org. Turns out the software was last updated about ten years ago. This meant that it didn't immediately work.
I managed to get it going, and decided to publicly host it on my git repository at https://kallithea.ethicallogistics.com/gallery2.py in case anyone else wants to use it.
It was a good exercise for me, as it is the first time I have been successful in fixing a python program by myself. Which is hopefully going to make it just that little bit easier to convert the forum in the future.
Here is the first gallery. Most of these photo's are already on the forum in different places. https://stuartcrawford.nz/pics/index.html
https://stuartcrawfordmedia.com/
https://nomadichome.org/
https://ethicallogistics.com/ Challenging the way you think.
JID:stuart@nomadicista.nz
https://nomadichome.org/
https://ethicallogistics.com/ Challenging the way you think.
JID:stuart@nomadicista.nz
- stuartcnz
- Site Admin
- Posts: 875
- Joined: Fri Dec 19, 2008 8:05 pm
- Location: Aotearoa, New Zealand
- Contact:
Re: Adventures in web community infrastructure
Well the server became disconnected from the internet again. Hopefully it will come back, like it did last time.
In the mean time, I have built a new server, and think I have the forum running again. The DNS should redirect to the new server in about 24 hours.
In the mean time, I have built a new server, and think I have the forum running again. The DNS should redirect to the new server in about 24 hours.
https://stuartcrawfordmedia.com/
https://nomadichome.org/
https://ethicallogistics.com/ Challenging the way you think.
JID:stuart@nomadicista.nz
https://nomadichome.org/
https://ethicallogistics.com/ Challenging the way you think.
JID:stuart@nomadicista.nz
- stuartcnz
- Site Admin
- Posts: 875
- Joined: Fri Dec 19, 2008 8:05 pm
- Location: Aotearoa, New Zealand
- Contact:
Re: Adventures in web community infrastructure
DNS seems to be working again. ie I am able to get to the forum via nomadicista.org.
https://stuartcrawfordmedia.com/
https://nomadichome.org/
https://ethicallogistics.com/ Challenging the way you think.
JID:stuart@nomadicista.nz
https://nomadichome.org/
https://ethicallogistics.com/ Challenging the way you think.
JID:stuart@nomadicista.nz
-
- Posts: 825
- Joined: Fri Apr 09, 2010 5:17 pm
- Location: Oklahoma
- Contact:
Re: Adventures in web community infrastructure
stuartcnz wrote:DNS seems to be working again. ie I am able to get to the forum via nomadicista.org.
It's an adventure...!!
- stuartcnz
- Site Admin
- Posts: 875
- Joined: Fri Dec 19, 2008 8:05 pm
- Location: Aotearoa, New Zealand
- Contact:
Re: Adventures in web community infrastructure
It certainly is an adventure At least I have learned enough from it, that I now have a database backup, that is never more than 24 hours old now. So recovering from a crash is no longer such a concern, just a hassle, as far as lost time for the forum, is concerned.
https://stuartcrawfordmedia.com/
https://nomadichome.org/
https://ethicallogistics.com/ Challenging the way you think.
JID:stuart@nomadicista.nz
https://nomadichome.org/
https://ethicallogistics.com/ Challenging the way you think.
JID:stuart@nomadicista.nz
- stuartcnz
- Site Admin
- Posts: 875
- Joined: Fri Dec 19, 2008 8:05 pm
- Location: Aotearoa, New Zealand
- Contact:
Re: Adventures in web community infrastructure
I received two backup copies of the database in my inbox this morning! It seems that the server that we were using a week ago has come back to life. Most odd, since I had powered it down.
It is good news though, as it means that when this server plays tricks (becomes un-contact-able) Then it should just be a matter of updating the database on the other server, and changing the DNS records, to point to it. Down time should only be a day then. And much less work for me to get it going again.
It is good news though, as it means that when this server plays tricks (becomes un-contact-able) Then it should just be a matter of updating the database on the other server, and changing the DNS records, to point to it. Down time should only be a day then. And much less work for me to get it going again.
https://stuartcrawfordmedia.com/
https://nomadichome.org/
https://ethicallogistics.com/ Challenging the way you think.
JID:stuart@nomadicista.nz
https://nomadichome.org/
https://ethicallogistics.com/ Challenging the way you think.
JID:stuart@nomadicista.nz
- stuartcnz
- Site Admin
- Posts: 875
- Joined: Fri Dec 19, 2008 8:05 pm
- Location: Aotearoa, New Zealand
- Contact:
Re: Adventures in web community infrastructure
And the new server went out!
We are back on the previous server now, having lost less than a full day, from the last backup. Unfortuantely, the last part day we did lose, happened to be the busiest time that the forum has had, in some time now.
I'm still working at getting the database converted to Postgresql, which would mean that the database could then be hosted on the production server, which has been proven to be fully reliable, since I started it. Then when these server issues come up again in the future (while we are still running the same forum software) the forum content will not be affected.
We are back on the previous server now, having lost less than a full day, from the last backup. Unfortuantely, the last part day we did lose, happened to be the busiest time that the forum has had, in some time now.
I'm still working at getting the database converted to Postgresql, which would mean that the database could then be hosted on the production server, which has been proven to be fully reliable, since I started it. Then when these server issues come up again in the future (while we are still running the same forum software) the forum content will not be affected.
https://stuartcrawfordmedia.com/
https://nomadichome.org/
https://ethicallogistics.com/ Challenging the way you think.
JID:stuart@nomadicista.nz
https://nomadichome.org/
https://ethicallogistics.com/ Challenging the way you think.
JID:stuart@nomadicista.nz
- stuartcnz
- Site Admin
- Posts: 875
- Joined: Fri Dec 19, 2008 8:05 pm
- Location: Aotearoa, New Zealand
- Contact:
Re: Adventures in web community infrastructure
Still no progress converting the database. I have decided to set up a new Linode server for the forum, for reliability in the mean time. I intend on having it available for at least 12 months, possibly 24 months. In which time I should be able to work out a better long time system.
I have also been running a social media site called Hubzilla, as well as my experimenting with Friendica, and have decided that I like Hubzilla, so will also set that up on the same Linode as the forum. Should happen within the next week or so.
I have also been running a social media site called Hubzilla, as well as my experimenting with Friendica, and have decided that I like Hubzilla, so will also set that up on the same Linode as the forum. Should happen within the next week or so.
https://stuartcrawfordmedia.com/
https://nomadichome.org/
https://ethicallogistics.com/ Challenging the way you think.
JID:stuart@nomadicista.nz
https://nomadichome.org/
https://ethicallogistics.com/ Challenging the way you think.
JID:stuart@nomadicista.nz
- stuartcnz
- Site Admin
- Posts: 875
- Joined: Fri Dec 19, 2008 8:05 pm
- Location: Aotearoa, New Zealand
- Contact:
Re: Adventures in web community infrastructure
I've just setup and security hardened a new Linode server. I intend on transferring the forum over in a couple of days, at which point there may be a four to six hour period of down time.
https://stuartcrawfordmedia.com/
https://nomadichome.org/
https://ethicallogistics.com/ Challenging the way you think.
JID:stuart@nomadicista.nz
https://nomadichome.org/
https://ethicallogistics.com/ Challenging the way you think.
JID:stuart@nomadicista.nz
Re: Adventures in web community infrastructure
Thanks for the heads up and all the work! Jack
- stuartcnz
- Site Admin
- Posts: 875
- Joined: Fri Dec 19, 2008 8:05 pm
- Location: Aotearoa, New Zealand
- Contact:
Re: Adventures in web community infrastructure
We're Back!
https://stuartcrawfordmedia.com/
https://nomadichome.org/
https://ethicallogistics.com/ Challenging the way you think.
JID:stuart@nomadicista.nz
https://nomadichome.org/
https://ethicallogistics.com/ Challenging the way you think.
JID:stuart@nomadicista.nz
Who is online
Users browsing this forum: No registered users and 10 guests