xen dev blog

[Sythe]

Going to miss our target launch date for xen I think. I've been working non stop on it even over Christmas. Matt has been doing a fair bit of work as well.

Database remains the major stumbling block. We use master-slave replication at the moment which has served us quite well. Xenforo doesn't natively support master-slave. There are addons but they are outdated and no longer maintained. Writing our own remains a possibility although master-slave is not a particularly good model to begin with.

I've been working with mysql cluster on the new servers to try provide a theoretically more stable platform that xen will run on, however having spent four straight days working with it now I'm thinking it's too fiddly for us. Essentially it is designed as a distributed memory database which is not appropriate for the size of our database. Even if I commissioned enough ram to hold our current database ( > 20Gb of ram ) it wouldn't allow for strong growth. It does have a disk storage option which I am testing at the moment but from what I've seen so far it is more cancer.

I haven't yet decided what to do. Mysql cluster was my first choice, and we may still go with it if I can figure out how to get disk storage to work cleanly and quickly. Failing that I think I will try mariadb cluster. Failing that we might try some sort of multi-master non-cluster setup.

Long story short: I'm working on it as quickly as I can but it's taking longer than expected. In the meantime I will continue to make critical patches to the existing production system. I am still planning to move us as soon as humanly possible.

 

[Wonderland]

I can confirm you and Matthew have been slaving at this for quite some time. I respect your commitment to Xenforo, and I think it's better to have everything running smoothly before switching over. Good luck with completing it.

 

[SuF]

Why is the database such a bottleneck? At work we have a 250GB+ SQLServer database that performs and scales just fine, other than running the dev machine out of disk space. Granted our use case is for a few very heavy users (requests on the order of 10s of MBs each) rather than many smaller requests.

 

[Sythe]

Why is the database such a bottleneck? At work we have a 250GB+ SQLServer database that performs and scales just fine, other than running the dev machine out of disk space. Granted our use case is for a few very heavy users (requests on the order of 10s of MBs each) rather than many smaller requests.

Different applications. We need high performance and redundancy together. It's not sufficient to have one really big server and run a single database instance on it. It would bottleneck no matter how expensive the hardware and doesn't provide a good failover option. Granted you could have two that check eachother but that's still not a great solution.

We need a scaling system made up of lots of reasonably small servers that work together -- shards. A system where we can add more capacity as needed without touching existing production hardware. The second you have to open hardware up you're talking serious major downtime and expense. If you can keep everything redundant with no reliance on any specific machine or piece hardware that's going to be the best overall option for uptime.

 

[Jack]

when is vb4 coming tho???

 

[Hamouze]

lick lick lick my ballssss

 

[Laurie]

I'm more concerned over security concerns than the new board, though saying that the upgrade with solve those issues, but I do welcome it with open arms, it's been a long time overdue.

Cheers big man.

 

[Jon]

Different applications. We need high performance and redundancy together. It's not sufficient to have one really big server and run a single database instance on it. It would bottleneck no matter how expensive the hardware and doesn't provide a good failover option. Granted you could have two that check eachother but that's still not a great solution.

We need a scaling system made up of lots of reasonably small servers that work together -- shards. A system where we can add more capacity as needed without touching existing production hardware. The second you have to open hardware up you're talking serious major downtime and expense. If you can keep everything redundant with no reliance on any specific machine or piece hardware that's going to be the best overall option for uptime.

You think working with OpenStack would benefit you, within your own dedicated environment?

 

[nodnarbusn]

Wait for vb4

 

[Brian]

xenforo is the bees knees fuck vb

 

[Xeroshadow48]

We need a scaling system made up of lots of reasonably small servers that work together -- shards.

I used to have some shards, if u need any just let me know and ill check my rs3 account

 

[Sythe]

I've burned mysql cluster. Looking at mariadb cluster today

 

[shawty]

try adding the http thread to the php network.
i had this problem before on an app i was developing. use this [HTTPOCTYPE//enter.format/938-382k.exception/addition.lok.39/end]

 

[MohtasaUnique]

 

[FlyingToast]

soon.

 

[ASAPgang]

Keep it going Rich, we appreciate the work.

 

[Sythe]

MariaDb cluster looks a lot more promising so far

 

[hattez]

Hope it works out for all of our sakes. Too many shall not pass getting put out there.

 

[Zing]

I'm more concerned over security concerns than the new board, though saying that the upgrade with solve those issues, but I do welcome it with open arms, it's been a long time overdue.

Cheers big man.

5char

 

[Brendan]

don't stress home slice, we'd rather it done properly than rushed