Webdock - Notice history

All systems operational

Denmark: Network Infrastructure - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 100.0%Jan 2025 · 100.0%
Nov 2024
Dec 2024
Jan 2025

Denmark: Storage Backend - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 100.0%Jan 2025 · 100.0%
Nov 2024
Dec 2024
Jan 2025

Denmark: General Infrastructure - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 100.0%Jan 2025 · 100.0%
Nov 2024
Dec 2024
Jan 2025

Canada: Network Infrastructure - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 100.0%Jan 2025 · 100.0%
Nov 2024
Dec 2024
Jan 2025

Canada: Storage Backend - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 100.0%Jan 2025 · 100.0%
Nov 2024
Dec 2024
Jan 2025

Canada: General Infrastructure - Operational

100% - uptime
Nov 2024 · 99.97%Dec · 100.0%Jan 2025 · 100.0%
Nov 2024
Dec 2024
Jan 2025

Webdock Statistics Server - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 100.0%Jan 2025 · 100.0%
Nov 2024
Dec 2024
Jan 2025

Webdock Dashboard - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 100.0%Jan 2025 · 100.0%
Nov 2024
Dec 2024
Jan 2025

Webdock Website - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 100.0%Jan 2025 · 100.0%
Nov 2024
Dec 2024
Jan 2025

Webdock Image Server - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 100.0%Jan 2025 · 100.0%
Nov 2024
Dec 2024
Jan 2025

Webdock REST API - Operational

100% - uptime
Nov 2024 · 100.0%Dec · 100.0%Jan 2025 · 100.0%
Nov 2024
Dec 2024
Jan 2025

Notice history

Jan 2025

Issue during power testing today, some customer VPS affected, Webdock services affected (including website)
  • Resolved
    Resolved

    All customer workloads have been operating for some time now. All Webdock services are now operational.

    Our electricians believe they have identified the issue and will be performing some maintenance related to this next week. The maintenance requires us to do another power outage test where there is still a chance that things may go as they did today where we loose power and customers will experience a restart. We of course hope this will not be the case.

    We will announce a maintenance window for this work at a later date.

  • Identified
    Identified

    Generator upgrade today went according to plan in most respects in that the new generator and power infrastructure performed as expected and our facility performed nominally. However, we identified an issue where a single rack lost all power as breakers were triggered during handover to generator power. The theory is that some equipment in that specific rack is faulty, causing a short circuit which took power.

    For this reason almost all Ryzen and Epyc workloads went offline for some minutes. Almost all services are up again but we are working through some snags in starting certain instances at the moment which we hope to resolve soon.

    We apologize for this incident and our electricians are hard at work trying to identify the malfunctioning equipment.

Dec 2024

Some issue with our webdock.cloud aliasdomain
  • Resolved
    Resolved

    As expected, the issue has now been resolved.

  • Update
    Update

    There is a workaround to the issue you can do if you need our webdock.cloud domain to resolve on your local machine (for example if you are developing something or want/need to connect using the aliasdomain, for example due to SSL certificates) - basically what you can do is edit your local hosts file to tell your system the IP of your domain, while we wait for the domain and DNS to become operational again. The guide can be seen here:

    https://webdock.io/en/docs/webdock-control-panel/ip-adresses/how-access-webdockcloud-if-dns-not-working

  • Update
    Update

    Unfortunately it turns out that the only team at our registrar that can reactivate us is Legal and they are based in New Zealand. This team only operates during business hours NZ time. This presumably means webdock.cloud will not become operational until sometime this evening CEST time or in about 9-12 hours worst case (as far as we can deduce). We have pressed their tech support as hard as we could, but they simply cannot do anything they say. We will be looking around for another domain registrar after this incident, as we simply cannot tolerate working with an organization which puts such an important function in a silo which is not available 24/7.

    We apologize for any inconvenience caused here today - in the meantime, you can always reach your server directly on your server IP address, or by pointing your own domain to your server.

  • Monitoring
    Monitoring

    We can now confirm that a rather silly thing happened which was that due to customer abuse (phishing) placed on our webdock.cloud aliasdomain - which was reported directly to our domain registrar (by somebody) and not direct to us, resulted in a suspension of our webdock.cloud domain. The notification of which went to our main admin email (our domain registrar does not allow for abuse contact to be entered with them, which is ... suboptimal) and that email of course ended up in a spam folder on our side, so we had no chance to react to it.

    We have cleared up the situation with them, but we are waiting on them to process the reactivation.

    We deal with abuse all the time and this has never happened before, so it is fortunately rare that abuse complaints go this circuitous (and wrong) route - but we have already made some changes which should ensure that our support/abuse team gets notified by our registrar if this ever happens again and have a chance to react in a timely manner.

    In the meantime all we can do now is wait for our registrar to reactivate the domain, hopefully soon.

  • Identified
    Identified

    We stopped seeing proper resolution of webdock.cloud this morning. We are looking into the status of our domain.

Network Maintenance in Denmark
  • Completed
    December 05, 2024 at 9:18 AM
    Completed
    December 05, 2024 at 9:18 AM

    After slow methodical work over the past two weeks we have completed all the network changes to our entire fleet in Denmark with no disruption to customers. Everything has been checked, double checked and triple checked :D We are happy to close this maintenance at last.

  • Update
    December 02, 2024 at 1:01 PM
    In progress
    December 02, 2024 at 1:01 PM
    Maintenance is now in progress.
  • In progress
    November 29, 2024 at 9:19 AM
    In progress
    November 29, 2024 at 9:19 AM

    We have been slowly pecking away at this in the safest manner possible, and for that reason only a small portion of our fleet in Denmark has been converted to the new configuration. We have decided to let this portion run until monday before doing the remainder and finally completing this maintenance. As stated before: This maintenance should have no impact on your server whatsoever.

  • Planned
    November 27, 2024 at 1:01 PM
    Planned
    November 27, 2024 at 1:01 PM

    We will be performing a rolling update of network configuration across our entire fleet in Denmark today and tomorrow. This update is to ensure the stability of our network and solve a long-standing issue whereby some customers have experienced sudden loss of connectivity due to routes beind dropped on our hosts. This update will eliminate this long-standing issue.

    Unfortunately, a subset of our customer instances need to be touched whereby our team adjust the network configuration inside your instance. This will typically only impact customers who have created servers with us recently - older instances and instances created within the last couple of weeks are unaffected and will not be touched.

    The operations usually just mean a reload of network configuration whereby - at worst - you will see a few packets lost and that's it. Only in rare cases where network configuration was already non-standard or bad beforehand in a customer instance will scenarios come up where you may see some disruption. We expect this to be minimal.

    Since this is a large task, we expect this maintenance to last throughout the day today and all day tomorrow, possibly longer depending on progress, in which case we will update here.

    Thank you for your patience and understanding.

  • In progress
    November 27, 2024 at 10:05 AM
    In progress
    November 27, 2024 at 10:05 AM

    This network maintenance has been on-going in stops and starts for the past week or so. We expect the final changes to be completed within the next 2 days. These changes should have NO impact on your server or workloads.

Nov 2024

Nov 2024 to Jan 2025

Next