xen dev blog

[Sythe]

I'll be making regular updates of what I'm doing in this thread for anyone who cares.

Currently loading test data onto an 8 node Percona cluster. Percona is a high performance variant of mariadb cluster.

Loading in the IP audit table which is 13 million rows and ~1.2Gib on MyISAM. To avoid running joins against this massive table the username of each entry (which is otherwise just userid, timestamp, IP and post/pmid) is encoded directly in the table which is why its so big. Further, char rather than varchar is used to make the table lookup really fast (fixed length row means instant indexing). This table is used in production as a lookup table when signing up users so it has to be quick.

The table has to be converted to InnoDB (or Percona's variant of it) to be loaded into the cluster. So far about 75% loaded and taking 2.1Gib on each node. To be expected, InnoDB takes more space.

Getting some 'failure to report' messages regarding committing every so often but the data appears to be writing successfully.

 

[Astro]

Good to hear no clue what all that tech talk is though. I hope it'll be a big improvement for sythe.org.

 

[Sythe]

Good news, the percona cluster took the test data without incident. An incomplete import of the site (converted for xen) is now running on the new cluster with a disk footprint of about 18gb. Expected final footprint to be about twice that.

Will do extensive testing tomorrow to ensure everything is going to work smoothly with this cluster solution before moving onto the remaining non-database work.

 

[Astro]

Good news, the percona cluster took the test data without incident. An incomplete import of the site (converted for xen) is now running on the new cluster with a disk footprint of about 18gb. Expected final footprint to be about twice that.

Will do extensive testing tomorrow to ensure everything is going to work smoothly with this cluster solution before moving onto the remaining non-database work.

May I ask what this Xen forum will do for sythe and what will change/improve?

 

[ASAPgang]

May I ask what this Xen forum will do for sythe and what will change/improve?

Gracious, the CDT are hoping to compile a guide/info for everyone in the shortcoming weeks.

 

[Sythe]

Gracious, the CDT are hoping to compile a guide/info for everyone in the shortcoming weeks.

I will be giving CDT early entry so they can help with transition

 

[ASAPgang]

I will be giving CDT early entry so they can help with transition

Thanks for that, Rich.

 

[Verts]

you wont regret the switch to xf

 

[nodnarbusn]

soon.

There's a lot of shit going on its just behind the scenes.

 

[Delta Squad]

lol toast

 

[Life2dmax]

Soon™

 

[hattez]

Anyone remember sythedice?

 

[Laurie]

remember what
?

 

[MohtasaUnique]

you wont regret the switch to xf

where have you been ,_,

 

[SuF]

where have you been ,_,

where have you been

 

[Sythe]

Spending quite a bit of time today debugging xen installation on the cluster

 

[Blupig]

diggin the spam forum updates bic rich-o

 

[Dunworry]

diggin the spam forum updates bic rich-o

Ikr? I too feel smart reading and pretending to follow along as if I understand it.

 

[Laurie]

i r excite

 

[FlyingToast]

lol toast

<3<3<3<3