Status updates on escalated tickets -- OVH, Hetzner

Not_Oles · September 2020

I have been a customer of OVH for a long time. My experience is that everything I've used at OVH works great almost all the time. For routine issues, OVH support has been prompt and helpful toward me.

Once every few years, however, some issue comes up which needs to be escalated. During the escalation period I have not found a good way to receive status reports. The first tier support people seem not to know what's happening "upstairs," and they do not seem eager to ask for updated status information, even for issues which seem to be affecting many customers. Of course, there is http://travaux.ovh.net/ but travaux is terse and sometimes not updated as often as one might like.

Does anybody know a good way to get escalation status reports from OVH? I mean, even if they just said something like "We haven't got around to this yet." at least customers would know what was happening. The people at OVH work hard and do a great job! I think OVH would be even better with the addition of detailed escalation status reports.

Does anybody know how Hetzner handles status reports when tickets are escalated? Does Hetzner have a page like travaux? @Hetzner_OL What about other top providers?

Edit to add: @willie @Falzo

seriesn · September 2020

With Ovh, you call them. That’s the only way to get “fast and quality “ support.

Not_Oles · September 2020

@seriesn said: only way

Too bad I want bare metal. It's the only way for me. Otherwise I could just sign up with you.

Maybe I will call OVH in the morning tomorrow if the problem isn't fixed by then.

seriesn · September 2020

@Not_Oles said:

@seriesn said: only way

Too bad I want bare metal. It's the only way for me. Otherwise I could just sign up with you.

One day my friend

Hetzner_OL · September 2020

@Not_Oles said: Does anybody know how Hetzner handles status reports when tickets are escalated? Does Hetzner have a page like travaux? @Hetzner_OL What about other top providers?

We have a status page. https://www.hetzner-status.de/ This is also quite similar to OVH's "travaux". The customer does not see a status of the tickets/support requests themselves. I hope this answers your question.

Falzo · September 2020

@seriesn said: With Ovh, you call them.

this. needed to do that only once in 15 years, if I remember correctly.

@Not_Oles said: Does anybody know how Hetzner handles status reports when tickets are escalated?

essentially as a customers it's not really a ticket, but mail conversation. works easy enough. I would call them though, if I ever think I would need to escalate ;-)

PS: never needed to do that with hetzner anyway, used their support only for hardware addons and such stuff.

vish · September 2020

@seriesn said:

@Not_Oles said:

@seriesn said: only way

Too bad I want bare metal. It's the only way for me. Otherwise I could just sign up with you.

One day my friend

This popped into my head lol

Xsltel · September 2020

btw calling OVH from Skype is free

Not_Oles · September 2020

A couple more days, apparently no further update to the travaux entry at http://travaux.ovh.net/?do=details&id=46690&

I finally filed my own ticket:

Subject: MX Plan - email not received
Number: 8001355435
Service: srvr.ovh
Date opened: 18/09/2020 15:37
Status: In progress
Latest modification: 18/09/2020 15:37

I have not yet called OVH on the phone.

Just to be clear -- I have very good feelings about OVH. They are terrific! They are honest, hard working, polite, and very helpful.

This particular issue is of special interest because there are no updates to the travaux page even though days are passing.

I am not sure that what seems to be a sending issue with the srvr.ovh MTA is even related to the issue mentioned on the travaux page.

So, for me, this thread is much, much more about "Updates are good" than "a possible problem at OVH."

Not_Oles · September 2020

The best example I know of great updating during a moment of huge trouble is the wonderful, heroic updates given by GitLab during their January 31, 2017 database outage. In case anybody is interested, I put some links below. Gitlab put up a Google Document which listed hourly or more updates. These updates not only tell what was happening but also report even the initials (and at one point, the full name of at least one) of the engineers who were working on the problem. Gitlab seemed completely transparent: the engineer who deleted the production database said who he was and how it had happened. In my book, that guy became a hero forever! I read the great news that the guy concerned was given a raise right after the incident. Last but not least, Gitlab apparently even was live streaming their attempt to repair the outage, though I personally haven't seen any of that stream.

The listing of people's names became a bit controversial. See the Hacker News discussion at the link below. Gitlab later backed off from names and used employee numbers.

Maybe live streaming is too much to expect? Maybe mentioning people by name is too much to expect? Maybe hourly reports are too much to expect?

Maybe daily updates are not too much to expect? For ongoing problems affecting multiple users. . . .

Maybe some update once in awhile might be nice?

Instead of just complaining, I wanted to share an inspiring example of how an internet hero posts updates. Thanks everyone!

Links:

Google search: Gitlab database outage
Gitlab Postmortem of database outage of January 31 (Feb 10, 2017)
Gitlab Blog post the day after it happened -- GitLab.com database incident (Feb 1, 2017)
Hacker News Discussion of Live Report GitLab Database Incident – Live Report
Google Docs page pointed to by HN page (now deleted)
Archive.org 318 saves of the now deleted Google Docs page
Archive.org earliest of the 318 saves -- February 2, 2017

Not_Oles · September 2020

@Hetzner_OL said:

@Not_Oles said: Does anybody know how Hetzner handles status reports when tickets are escalated? Does Hetzner have a page like travaux? @Hetzner_OL What about other top providers?

We have a status page. https://www.hetzner-status.de/ This is also quite similar to OVH's "travaux". The customer does not see a status of the tickets/support requests themselves. I hope this answers your question.

Hi @Hetzner_OL! Thanks for the answer to my question! I looked at the Status page. I'm so glad it's there!

Not_Oles · September 2020

I meant to have one thread about the srvr.ovh email outage and another about how providers disclose what happens while issues are being addressed.

I ended up with two mixed threads. My bad! Sorry for the extra noise.

The good news that the srvr.ovh email failure-to-send issue has been fixed may be found at https://talk.lowendspirit.com/discussion/1780/ovh-and-srvr-ovh-email-outage Yaaay!

If anybody has another example of great disclosure during a problem -- if there even is another example comparable to the Gitlab incident mentioned above -- I'd really appreciate your posting it here, please.

Greetings from Mexico's Sonoran Desert!

xreann20 · September 2020

Offtopic but I want to ask @Not_Oles if you know about the internal DDoS issue, has it been fixed? When I was with OVH hosting a game server there was a lot of internal DDoS that passes through their firewall since its internal traffic.

Not_Oles · September 2020

@xreann20 said:
Offtopic but I want to ask @Not_Oles if you know about the internal DDoS issue, has it been fixed? When I was with OVH hosting a game server there was a lot of internal DDoS that passes through their firewall since its internal traffic.

I see the internal DDoS issue mentioned frequently. I am unaware of any fix. I certainly am unaware of any general commenting about some kind of fix having happened.

My own experience with OVH has been pretty good. And for quite awhile now, maybe six or seven years. Of course, I'm definitely not Oles, so I have no access to inside information. But it's really hard for me to imagine that OVH would fail to do something effective about any issue they regard as significant.

Not_Oles · September 2020

Nobody else has an example of great disclosure and updates?

xreann20 · September 2020

@Not_Oles said: I see the internal DDoS issue mentioned frequently. I am unaware of any fix. I certainly am unaware of any general commenting about some kind of fix having happened.

My own experience with OVH has been pretty good. And for quite awhile now, maybe six or seven years. Of course, I'm definitely not Oles, so I have no access to inside information. But it's really hard for me to imagine that OVH would fail to do something effective about any issue they regard as significant.

I see it not being fixed for a long time, I was hoping they fix it in time of around a year, then after 1.5 years of almost every other week of internal attacks, tired of capturing IPs and giving it to them, no workaround is done. That is why I switched, their service is good enough especially the pricing and specs, but that was my major issue they cannot solve. Anyway thanks.

Not_Oles · September 2020

@xreann20 said: I switched

Might you want to share to whom you switched? By what method does your new provider handle the DDoS issues?

I, too, always need help.

xreann20 · September 2020

@Not_Oles said: Might you want to share to whom you switched? By what method does your new provider handle the DDoS issues?

I, too, always need help.

I switched to @splitice , tho I only need their service for my gameservers, its quite helpful. OVH DDoS protection if very good unless the attack comes from their network, that is their biggest weakness, it can take down your server easily as the traffic does not go thru VAC and directly hit your servers.

Not_Oles · September 2020

Funny how when I start to think about things, something relevant tends to pop up on Hacker News.

This morning there was a post to HN sharing a how to blog for communicating with customers during incidents: https://news.ycombinator.com/item?id=24617918

I think it's interesting that nobody here has posted a response to my request for additional examples of great incident response. I haven't yet tried directly researching this topic. I imagine finding a whole community of public relations incident response professionals. The blog post at which the above HN post pointed has a few links at the end.

I remember there was a Gandi outage awhile back where they had some pretty good status updates during and after the event. . . . .

Any help finding a couple more examples of great incident response would be much appreciated. Thanks! Greetings from Sonora!

bikegremlin · September 2020

@Not_Oles said:
Funny how when I start to think about things, something relevant tends to pop up on Hacker News.

This morning there was a post to HN sharing a how to blog for communicating with customers during incidents: https://news.ycombinator.com/item?id=24617918

I think it's interesting that nobody here has posted a response to my request for additional examples of great incident response. I haven't yet tried directly researching this topic. I imagine finding a whole community of public relations incident response professionals. The blog post at which the above HN post pointed has a few links at the end.

I remember there was a Gandi outage awhile back where they had some pretty good status updates during and after the event. . . . .

Any help finding a couple more examples of great incident response would be much appreciated. Thanks! Greetings from Sonora!

If it's of any help - MDDhosting:

https://forums.mddhosting.com/topic/1582-major-outage-092118-09242018/

https://www.webhostingtalk.com/showthread.php?t=1731410

Not_Oles · September 2020

If it's of any help

@bikegremlin Very helpful! Thank you! I read the pages you linked.

Status updates on escalated tickets -- OVH, Hetzner

Comments

This Site is currently in maintenance mode.
Please check back here later.

Status updates on escalated tickets -- OVH, Hetzner

Comments

This Site is currently in maintenance mode.Please check back here later.

This Site is currently in maintenance mode.
Please check back here later.