Cluster issues

Apologies for the unexpected downtime

I was rebuilding the IP audit table and managed to crash the cluster which in turn revealed we were using the wrong state transfer mechanism. Should be all better in a minute.

 

[Tyler]

got em coach

 

Back up for me, cheers!

 

working just fine now for me,

 

Only 1/8 cluster nodes is up. I'm syncing a second. Then I can sync 2 more from there and the remaining four from there.

 

Apology accepted

 

alright after much pain, we now have 2 of 8 up lol

the others should be easy now we have the right state transfer configured

undocumented percona rubbish you actually have to comment out the whole wsrep_sst_method line to get xtrabackup to work properly... what even

 

we're at 3 of 8

just bringing them up individually at the moment to avoid any more problems

 

4/8

 

5/8

 

6/8

 

7/8

 

8/8

ok back to rebuilding IP audit system hopefully this time without crashing the cluster lol

 

[Tyler]

Thanks for updating everytime you fixed a cluster @Sythe

Its important we know this information

 

Thanks for updating everytime you fixed a cluster @Sythe

Its important we know this information

I thought so as well. Misleading to do it 1 by 1 when you said 2/8 -> 4/8 -> 8/8 though, why lie I don't get it

 

Thanks for updating everytime you fixed a cluster @Sythe

Its important we know this information

I thought so as well. Misleading to do it 1 by 1 when you said 2/8 -> 4/8 -> 8/8 though, why lie I don't get it

didn't want to push it offline again so I took it slow

You may not realize but the cluster is not in a healthy/stable state until all 8 nodes are up that's why I was updating you.

 

230,000 row IPaudit patch is now being applied

 

didn't want to push it offline again so I took it slow

You may not realize but the cluster is not in a healthy/stable state until all 8 nodes are up that's why I was updating you.

It was a meme lol sorry nobody can make memes as good as your dat boi game...

 

O shit waddup

 

Ok IP audit patching is all done

time to see if its worked