Adventures in web community infrastructure

For anything that doesn't fit the other forums.

Moderator: TMAX

ol trunt
Posts: 551
Joined: Thu Jan 17, 2013 2:51 pm
Location: Southern California
Contact:

Re: Adventures in web community infrastructure

Post by ol trunt »

Ditto what Dusty said! Jack
User avatar
stuartcnz
Site Admin
Posts: 875
Joined: Fri Dec 19, 2008 8:05 pm
Location: Aotearoa, New Zealand
Contact:

Re: Adventures in web community infrastructure

Post by stuartcnz »

Among the many, one of the big challenges is going to be, to convert the existing database from MySQL, to PostgreSQL. The reason for that is that the server that I intend on putting the new forum on, already has a running PostgreSQL server. It doesn't make sense to try and also run a MySQL server on it as well. As far as I know, both the existing forum software, and what I want to convert to, are happy running on either of the two types of database. I'm not so sure that converting the database itself, is so straight forward though.
User avatar
stuartcnz
Site Admin
Posts: 875
Joined: Fri Dec 19, 2008 8:05 pm
Location: Aotearoa, New Zealand
Contact:

Re: Adventures in web community infrastructure

Post by stuartcnz »

In order to figure out how to convert the MySQL database to PostgreSQL, I have set up a virtual machine on my desktop computer. I have installed the forum software and imported a recent snapshot of the forum database.

The first tool that I tried to do the conversion, is called pgloader. So far I have not been able to get it to work. My next trick will be to try another tool called FromMySqlToPostgreSql which, from what I can tell, is a PHP script, using a configuration file, customized for the particular database to be converted.

If I can get that to work, and can verify that it works with the forum software on the virtual machine, then it would be time to convert the forum database, to run on Postgres.
Dusty
Posts: 356
Joined: Sat Jan 09, 2010 6:06 am
Location: New Zealand
Contact:

Re: Adventures in web community infrastructure

Post by Dusty »

Phew :?
58 Bedford House
User avatar
stuartcnz
Site Admin
Posts: 875
Joined: Fri Dec 19, 2008 8:05 pm
Location: Aotearoa, New Zealand
Contact:

Re: Adventures in web community infrastructure

Post by stuartcnz »

A bit of a side track, but part of the process: I wanted to be able to host my own photo's for posting on the forum, and possibly other places. I have just been sticking them in the back of other sites, but it was both a bit of a hassle to do, and a bit messy.

So I have been hunting around for something simple, and written in Python to do it, and preferably without needing to run a database for it.

Last night, I found an example that looked good, but couldn't access the site, that hosts the code. This morning I managed to get a copy through archive.org. Turns out the software was last updated about ten years ago. This meant that it didn't immediately work.

I managed to get it going, and decided to publicly host it on my git repository at https://kallithea.ethicallogistics.com/gallery2.py in case anyone else wants to use it.

It was a good exercise for me, as it is the first time I have been successful in fixing a python program by myself. Which is hopefully going to make it just that little bit easier to convert the forum in the future.

Here is the first gallery. Most of these photo's are already on the forum in different places. https://stuartcrawford.nz/pics/index.html
User avatar
stuartcnz
Site Admin
Posts: 875
Joined: Fri Dec 19, 2008 8:05 pm
Location: Aotearoa, New Zealand
Contact:

Re: Adventures in web community infrastructure

Post by stuartcnz »

Well the server became disconnected from the internet again. Hopefully it will come back, like it did last time.

In the mean time, I have built a new server, and think I have the forum running again. The DNS should redirect to the new server in about 24 hours.
User avatar
stuartcnz
Site Admin
Posts: 875
Joined: Fri Dec 19, 2008 8:05 pm
Location: Aotearoa, New Zealand
Contact:

Re: Adventures in web community infrastructure

Post by stuartcnz »

DNS seems to be working again. ie I am able to get to the forum via nomadicista.org.
Stealth Camper
Posts: 824
Joined: Fri Apr 09, 2010 5:17 pm
Location: Oklahoma
Contact:

Re: Adventures in web community infrastructure

Post by Stealth Camper »

stuartcnz wrote:DNS seems to be working again. ie I am able to get to the forum via nomadicista.org.

It's an adventure...!!
User avatar
stuartcnz
Site Admin
Posts: 875
Joined: Fri Dec 19, 2008 8:05 pm
Location: Aotearoa, New Zealand
Contact:

Re: Adventures in web community infrastructure

Post by stuartcnz »

It certainly is an adventure :D At least I have learned enough from it, that I now have a database backup, that is never more than 24 hours old now. So recovering from a crash is no longer such a concern, just a hassle, as far as lost time for the forum, is concerned.
User avatar
stuartcnz
Site Admin
Posts: 875
Joined: Fri Dec 19, 2008 8:05 pm
Location: Aotearoa, New Zealand
Contact:

Re: Adventures in web community infrastructure

Post by stuartcnz »

I received two backup copies of the database in my inbox this morning! It seems that the server that we were using a week ago has come back to life. Most odd, since I had powered it down.
It is good news though, as it means that when this server plays tricks (becomes un-contact-able) Then it should just be a matter of updating the database on the other server, and changing the DNS records, to point to it. Down time should only be a day then. And much less work for me to get it going again.
User avatar
stuartcnz
Site Admin
Posts: 875
Joined: Fri Dec 19, 2008 8:05 pm
Location: Aotearoa, New Zealand
Contact:

Re: Adventures in web community infrastructure

Post by stuartcnz »

And the new server went out! :banghead:
We are back on the previous server now, having lost less than a full day, from the last backup. Unfortuantely, the last part day we did lose, happened to be the busiest time that the forum has had, in some time now. :banghead:

I'm still working at getting the database converted to Postgresql, which would mean that the database could then be hosted on the production server, which has been proven to be fully reliable, since I started it. Then when these server issues come up again in the future (while we are still running the same forum software) the forum content will not be affected.
User avatar
stuartcnz
Site Admin
Posts: 875
Joined: Fri Dec 19, 2008 8:05 pm
Location: Aotearoa, New Zealand
Contact:

Re: Adventures in web community infrastructure

Post by stuartcnz »

Still no progress converting the database. I have decided to set up a new Linode server for the forum, for reliability in the mean time. I intend on having it available for at least 12 months, possibly 24 months. In which time I should be able to work out a better long time system.
I have also been running a social media site called Hubzilla, as well as my experimenting with Friendica, and have decided that I like Hubzilla, so will also set that up on the same Linode as the forum. Should happen within the next week or so.
User avatar
stuartcnz
Site Admin
Posts: 875
Joined: Fri Dec 19, 2008 8:05 pm
Location: Aotearoa, New Zealand
Contact:

Re: Adventures in web community infrastructure

Post by stuartcnz »

I've just setup and security hardened a new Linode server. I intend on transferring the forum over in a couple of days, at which point there may be a four to six hour period of down time.
ol trunt
Posts: 551
Joined: Thu Jan 17, 2013 2:51 pm
Location: Southern California
Contact:

Re: Adventures in web community infrastructure

Post by ol trunt »

Thanks for the heads up and all the work! Jack
User avatar
stuartcnz
Site Admin
Posts: 875
Joined: Fri Dec 19, 2008 8:05 pm
Location: Aotearoa, New Zealand
Contact:

Re: Adventures in web community infrastructure

Post by stuartcnz »

We're Back!
Post Reply

Who is online

Users browsing this forum: No registered users and 3 guests