Status updates | Instruqt

Tracks with GCP cloud accounts may fail to provision

Wed, 16 Sep 2026 13:41:00 -0000

Tracks with GCP cloud accounts are provisioning normally. Between 05:33 and 10:45 UTC on 16 September 2026, a portion of the tracks including GCP cloud account failed to provision because Google Cloud intermittently returned an internal error while enabling the Compute Engine API on newly created projects. We apologize for the inconvenience this caused. We are improving our provisioning process to handle this type of transient Google Cloud error more reliably, reducing the likelihood that it will affect track starts in the future.

Tracks with GCP cloud accounts may fail to provision

Wed, 16 Sep 2026 05:33:00 -0000

Tracks that include a GCP cloud account may fail to provision. Affected learners see a setup error shortly after starting the track; starting the track again usually succeeds. Tracks without a GCP cloud account are unaffected, as are AWS and Azure cloud accounts. The cause is an intermittent internal error returned by Google Cloud while enabling APIs on a newly created project. We are improving our provisioning flow to be more resilient against this case and have raised the error with Google Cloud support.

Some sandboxes with VMs are failing to start

Fri, 04 Sep 2026 20:23:00 -0000

### Resolved The Google Cloud incident affecting multiple products in the `us-central1` region has been resolved. According to Google Cloud, the issue was fully resolved for all affected users as of **4 September 2026 at 13:06 PDT (20:06 UTC)**. Their preliminary root cause analysis indicates that the incident was caused by a faulty change, which was rolled back to mitigate the issue. We have also **re-enabled `us-central1` for sandbox provisioning** and restored our normal provisioning configuration. We will work on improving the resilience of our sandbox provisioning process against this type of failure in the future. We apologize for the inconvenience and appreciate your patience throughout the incident.

Some sandboxes with VMs are failing to start

Fri, 04 Sep 2026 16:58:00 -0000

### Update The issue is still ongoing. Below is the latest update from Google Cloud, published on 4 September 2026 at 18:26:39 UTC+2: > We are investigating an issue with Google Compute Engine, Persistent Disk, Google Cloud Dataflow, Cloud SQL, Google Kubernetes Engine and Cloud Filestore in `us-central1` region. > > We are experiencing an issue with Google Compute Engine control plane operations (VM creations, deletion, etc) beginning on Friday, 2026-09-04 06:39 PDT. Existing VMs are not affected. Cloud Filestore, Google Kubernetes Engine, Cloud Dataflow are affected. > > Our engineering team is attempting a mitigation and continues to investigate the issue. > > We will provide an update by Friday, 2026-09-04 10:30 PDT with details. We continue to track the incident and will share further updates as the situation develops.

Some sandboxes with VMs are failing to start

Fri, 04 Sep 2026 15:30:00 -0000

Observability data indicates that the issues on the Google Cloud side started escalating again around 15:05 UTC. To prevent continued failures, we have temporarily disabled `us-central1` for sandbox provisioning. During this period, sandboxes for manual plays and hot start pools configured for the `us-central1` region will be provisioned in `europe-west1` instead. We apologize for the inconvenience and appreciate your patience while we work to mitigate the impact. We will continue monitoring the situation closely and provide further updates as the status changes.

Some sandboxes with VMs are failing to start

Fri, 04 Sep 2026 14:25:00 -0000

One of the cloud zones (`us-central1-a`) where we provision virtual machines is currently experiencing issues. While most machines are still being provisioned within a few seconds, some requests are getting stuck and eventually hitting the timeout threshold. This primarily affects learners accessing the platform close to the `us-central1` region, and the scope is hot start pools referenced to a track or sandbox preset with a VM. The issue started at 13:17 UTC. It subsided around 13:58 UTC but returned at 14:15 UTC, so the impact has been intermittent rather than a continuous outage. We apologize for any inconvenience and appreciate your patience as we continue working with Google Cloud Support to resolve the issue.

Sandbox start failures for tracks using images hosted on Google Container Registry

Mon, 31 Aug 2026 11:53:00 -0000

Google Cloud has confirmed an incident affecting Artifact Registry in the US multiregion, which is where some of the container images used by Instruqt sandboxes are hosted. Google reports the issue was resolved for all affected users at 11:21 UTC on 31 August 2026. The sandbox start failures we observed earlier today are consistent with this incident. During the affected period, some tracks and labs failed to start when a sandbox host referenced a container image hosted on gcr.io, including Instruqt-provided images such as gcr.io/instruqt/cloud-client. Our last observed failure was at 11:17 UTC. Image lookups have been operating normally since. No action is needed on your side, and tracks that failed to start can be started again.

Sandbox start failures for tracks using images hosted on Google Container Registry

Mon, 31 Aug 2026 08:30:00 -0000

Since roughly 08:30 UTC, some tracks and labs have failed to start when a sandbox host references a container image hosted on Google Container Registry (gcr.io), including Instruqt-provided images such as gcr.io/instruqt/cloud-client. Affected sessions fail during sandbox creation rather than partway through. The failures are intermittent. Retrying a track start will often succeed, and no configuration change is needed on your side. Tracks referencing images from other registries do not appear to be affected. We have raised the issue with Google Cloud support and are working with them to identify the cause. We will update this notice as we learn more.

Maintenance: Database Maintenance

Mon, 24 Aug 2026 13:27:33 -0000

Scheduled Maintenance Notice We will be performing routine maintenance on September 6, 2026, starting at 02:00 (UTC). The maintenance is expected to last approximately 1 hour. During this time, you may experience brief errors or interruptions in service. We apologize for any inconvenience and appreciate your patience as we continue to improve our services.

Maintenance: Database Maintenance

Tue, 09 Jun 2026 07:52:08 -0000

Scheduled Maintenance Notice We will be performing routine maintenance on June 28, 2026, starting at 02:00 (UTC). The maintenance is expected to last approximately 1 hour. During this time, you may experience brief errors or interruptions in service. We apologize for any inconvenience and appreciate your patience as we continue to improve our services.

Sandbox Provisioning - europe-west1 Region Temporarily Disabled

Tue, 26 May 2026 11:30:00 -0000

europe-west1 Sandbox Provisioning Restored We are happy to inform you that the issue affecting sandbox provisioning in the europe-west1 region has been resolved. The region is now fully operational and sandboxes will resume provisioning in europe-west1 as expected for both manual and hot start plays. We apologize for any inconvenience this may have caused and thank you for your patience.

Sandbox Provisioning - europe-west1 Region Temporarily Disabled

Tue, 26 May 2026 10:30:00 -0000

Due to an ongoing issue on Google Cloud Platform's side, we are currently unable to provision sandboxes in the europe-west1 region. We have disabled this region while the support case is being resolved. Impact: Sandboxes for both manual and hot start plays that would normally be provisioned in europe-west1 will automatically be provisioned in us-central1 until the issue is resolved. We are actively working with GCP support to restore europe-west1 availability and will provide updates as the situation progresses.

Sandbox Provisioning - europe-west1 Region Temporarily Disabled

Wed, 20 May 2026 14:16:00 -0000

Sandbox Provisioning - europe-west1 Region Temporarily Disabled

Wed, 20 May 2026 11:30:00 -0000

Degraded Performance

Wed, 06 May 2026 20:23:00 -0000

The performance issue affecting lab start times has been resolved. All services are operating normally. Thank you for your patience.

Degraded Performance

Wed, 06 May 2026 19:28:00 -0000

We are currently experiencing degraded performance which may result in slower lab start times. We will provide updates as more information becomes available.

Maintenance: Database Maintenance

Tue, 21 Apr 2026 07:36:01 -0000

Scheduled Maintenance Notice We will be performing routine maintenance on May 10, 2026, starting at 02:00 (UTC). The maintenance is expected to last approximately 1 hour. During this time, you may experience brief errors or interruptions in service. We apologize for any inconvenience and appreciate your patience as we continue to improve our services.

Maintenance: Database maintenance

Fri, 03 Apr 2026 12:28:58 -0000

Scheduled Maintenance Notice We will be performing scheduled maintenance on Sunday, April 5, 2026, starting at 08:30 UTC, to enhance our observability capabilities. The maintenance is expected to take approximately 15 minutes. During this time, you may experience brief interruptions or errors, particularly affecting the reporting feature. We apologize for any inconvenience and appreciate your patience as we continue to improve our services.

Error in track provisioning in europe-west1

Tue, 24 Mar 2026 14:53:00 -0000

We're seeing that GCP Cloud DNS response latencies in europe-west1 have returned to normal levels, and sandbox provisioning in the region has been re-enabled. We will continue to monitor the situation closely and take appropriate action if the issue reoccurs. We sincerely apologize for the inconvenience and appreciate your patience while we worked to resolve this issue.

Error in track provisioning in europe-west1

Tue, 24 Mar 2026 14:24:00 -0000

We have isolated the issue to CloudDNS provisioning delays. We are seeing some records being created, however at this time it is still inconsistent. We have applied a workaround to disable the europe-west1 region. This may result in latency during plays as sandboxes will be provisioned in us-central1 and asia-southeast1.

Error in track provisioning in europe-west1

Tue, 24 Mar 2026 13:56:00 -0000

We are currently experience provisioning issues in europe-west1 region for Virtual Machines and Containers. Our engineers are actively looking into this. Sandboxes in Hot Start pools started more than an hour ago should not be affected.

Elevated error rate in VM creation requests (us-central1)

Fri, 06 Mar 2026 07:28:00 -0000

The issue was confirmed and resolved by Google Engineering team at 2026-03-05 22:30 UTC. The last errors we received were one hour prior at 21:30 UTC. To confirm, less than 0.1% of plays were affected. The root cause according to the Google Engineers was due to an internal operation reaching an internal quota in the us-central1-a zone. The issue was mitigated by increasing the internal quota. We will continue to monitor for these specific issues going forward.

Elevated error rate in VM creation requests (us-central1)

Thu, 05 Mar 2026 12:21:00 -0000

We continue to see some failures today with sandbox provisioning (<0.1% of plays in the last 8 hours). We are in contact with Google support to understand the root cause of the issue. We continue to monitor the situation, and will update if there are significant changes in error rates.

Elevated error rate in VM creation requests (us-central1)

Wed, 04 Mar 2026 12:50:00 -0000

We are currently experiencing a higher-than-normal error rate for VM creation requests in us-central1 region. This may result in intermittent provisioning failures. We are actively working with Google to investigate and resolve the issue. Workaround: In the meantime, we recommend using Hot Starts, which include automatic retries on provisioning failures. We will provide further updates as soon as more information becomes available.

Maintenance: Database Maintenance

Sun, 01 Feb 2026 15:17:28 -0000

Scheduled Maintenance Notice We will be performing routine maintenance on February 15, 2026, starting at 02:00 (UTC). The maintenance is expected to last approximately 1 hour. During this time, you may experience brief errors or interruptions in service. We apologize for any inconvenience and appreciate your patience as we continue to improve our services.

Partial Service Degradation: Indirect Impact from Cloudflare Outage

Fri, 05 Dec 2025 09:19:00 -0000

The upstream Cloudflare incident has been resolved. - Docker Hub: Image pulls are now successful. - Public Docs (GitBook): Documentation is fully accessible. All indirectly affected services are confirmed as restored. We are currently in a monitoring phase to confirm full stability across all systems.

Partial Service Degradation: Indirect Impact from Cloudflare Outage

Fri, 05 Dec 2025 08:48:00 -0000

We are monitoring a major Cloudflare outage that is indirectly affecting some of our external dependencies. Our core services are operational, but the following third-party components are currently inaccessible: - Docker Hub: Users cannot pull images - Public Documentation (GitBook): Our public documentation is inaccessible. We are actively monitoring the upstream provider. We will provide an update when the external services recover. You can access the upstream status here: https://www.cloudflarestatus.com/

Maintenance: Database Maintenance

Sun, 09 Nov 2025 14:27:50 -0000

Scheduled Maintenance Notice We will be performing routine maintenance on November 16, 2025, starting at 02:00 (UTC). The maintenance is expected to last approximately 1 hour. During this time, you may experience brief errors or interruptions in service. We apologize for any inconvenience and appreciate your patience as we continue to improve our services.

Azure Cloud Account Provisioning Failure

Thu, 30 Oct 2025 15:26:00 -0000

Azure's earlier flaky performance has now fully recovered, and sandbox provisioning with Azure cloud accounts is back to normal. We've also deployed a change to better tolerate similar issues in the future.

Azure Cloud Account Provisioning Failure

Thu, 30 Oct 2025 14:00:00 -0000

We're observing improvements in Azure's performance and sandbox provisioning with Azure cloud accounts has started to recover. Our team continues to monitor the situation closely. In parallel, we're implementing a change to make our systems more resilient to similar Azure issues in the future.

Azure Cloud Account Provisioning Failure

Thu, 30 Oct 2025 12:30:00 -0000

We're currently experiencing issues with provisioning sandboxes including Azure cloud accounts. Our team is actively investigating and will provide updates as soon as more information becomes available.

Sandbox provisioning issues

Wed, 22 Oct 2025 11:07:00 -0000

We are no longer observing intermittent IAM permission errors from Google Cloud Platform (GCP). The issue appears to have been resolved, and we have fully restored the Host Images feature. Sandbox provisioning and related functionality are now operating normally.

Docker Hub Disruption Impacting Sandbox Provisioning

Tue, 21 Oct 2025 10:47:00 -0000

Docker Hub has recovered from the earlier disruption caused by the widespread AWS outage. Sandboxes containing a container host that pulls images from Docker Hub are now provisioning successfully. We will continue to monitor the situation, but no further impact is expected.

HubSpot and Salesforce Integrations Disruption

Tue, 21 Oct 2025 10:39:00 -0000

Our third-party vendor Paragon has fully recovered from the earlier service disruption. The Integrations page and event submissions to Salesforce and HubSpot are now functioning as expected. We will continue to monitor to ensure stability.

Sandbox provisioning issues

Tue, 21 Oct 2025 10:34:00 -0000

Sandbox provisioning is now functioning as expected. To restore normal operations, we temporarily disabled the "Host Images" feature. Our team is actively working to resolve the issue and re-enable this functionality as soon as possible.

Sandbox provisioning issues

Tue, 21 Oct 2025 03:00:00 -0000

We are currently investigating an issue affecting sandbox provisioning. Users may experience failures or delays when creating new sandboxes. Our team is actively working to identify the root cause and will provide an update once more information becomes available.

AWS Cloud Account Provisioning Failure

Mon, 20 Oct 2025 14:40:00 -0000

AWS has recovered from the widespread outage. Tracks including AWS Cloud accounts are now provisioning successfully. We will continue monitoring to ensure stability, but no further impact is expected.

Docker Hub Disruption Impacting Sandbox Provisioning

Mon, 20 Oct 2025 13:25:00 -0000

Due to the widespread AWS outage, Docker Hub is currently experiencing service disruptions. As a result, tracks using container sandbox hosts that pull images from Docker Hub may encounter increased latency or provisioning failures. We are closely monitoring the situation and will provide updates as soon as there are any changes.

HubSpot and Salesforce Integrations Disruption

Mon, 20 Oct 2025 11:38:00 -0000

Our third-party vendor, Paragon, which powers our integrations, is currently experiencing a service disruption due to a widespread AWS outage. During this period, the Integrations page may fail to load, and event submissions to Salesforce and HubSpot may not be possible. We are actively monitoring the situation and will provide updates as soon as conditions change.

Azure Portal login enforces MFA, causing login issues

Tue, 07 Oct 2025 07:30:00 -0000

We have been monitoring the deployed fix to avoid the MFA login requirement for Azure over the weekend and confirmed it is working as expected.

Azure Portal login enforces MFA, causing login issues

Fri, 03 Oct 2025 14:37:00 -0000

We have identified and deployed a fix to avoid the MFA login requirement for Azure. We will monitor this functionality through the weekend. We do not expect track plays to be impacted, however we are leaving this marked as Degraded out of caution.

Azure Portal login enforces MFA, causing login issues

Thu, 02 Oct 2025 07:24:00 -0000

Azure recently started enforcing MFA through their Authenticator app. This affects logging into the Azure Portal using our temporary credentials. The MFA requirement cannot be turned off. We are working on a solution. We will post an update by 2025-10-03 14:00Z latest.

Maintenance: Database Maintenance

Thu, 18 Sep 2025 12:05:31 -0000

Scheduled Maintenance Notice We will be performing routine maintenance on September 28, 2025, starting at 02:00 (UTC). The maintenance is expected to last approximately 1 hour. During this time, you may experience brief errors or interruptions in service. We apologize for any inconvenience and appreciate your patience as we continue to improve our services.

Maintenance: Upcoming Proxy Maintenance

Wed, 27 Aug 2025 12:09:49 -0000

We will be performing maintenance on proxy instances that provide interface to the sandbox hosts. During this time, you may experience brief connectivity interruptions with the following: - Terminal - Code Editor - Service / Web Application - Virtual Browser / Web Page Running sandboxes and new sandbox creation will not be affected. Thank you for your patience as we continue to improve our services.

Intermittent SSL Certificate Provisioning Failures

Thu, 07 Aug 2025 08:35:00 -0000

We're seeing that DNS-related metrics have returned to normal, and SSL certificate provisioning is functioning reliably again. While the immediate symptoms have subsided, our support case with the third-party provider remains open as we continue to monitor and validate stability, but at this time, no further customer impact is expected. Thank you for your patience while we worked through this issue.

Intermittent SSL Certificate Provisioning Failures

Wed, 06 Aug 2025 13:09:00 -0000

We're currently experiencing sporadic SSL certificate provisioning issues due to instability with a third-party DNS provider we rely on for required DNS record creation. We have an active support case open with the provider and are closely monitoring the situation. In parallel, we're also evaluating alternative solutions to mitigate the impact and improve reliability moving forward. We sincerely apologize for the inconvenience and appreciate your patience as we work to resolve this.

Application unavailable

Wed, 23 Jul 2025 08:53:00 -0000

The issue has been isolated and services return to normal health. Our apologies for any inconvenience

Application unavailable

Wed, 23 Jul 2025 08:22:00 -0000

Our application is down right now - engineers are currently investigating.

Maintenance: Database Maintenance

Tue, 15 Jul 2025 09:37:30 -0000

We will be performing a database upgrade on Sunday, 3rd August 2025, starting at 07:00:00 (UTC) until 10:00:00 (UTC). Our application is expected to be unavailable at times inside of this window. We will post an In maintenance page for anyone attempting to play any tracks whilst the database is being upgraded. Per our planning, we do not expect to need the entire maintenance window, however the window reflects our worst case scenario. We apologize for any inconvenience and appreciate your patience as we continue to improve our services.

Google Cloud outage

Thu, 12 Jun 2025 20:30:00 -0000

All services are restored and healthy

Google Cloud outage

Thu, 12 Jun 2025 20:05:00 -0000

We are currently seeing continued issues affecting Instruqt track plays for users based in the USA, as Google Cloud Platform (GCP) services in the USA are still in the process of recovering. Users accessing Instruqt from outside the USA are generally seeing services operate as normal. We are actively monitoring the situation and will provide further updates as GCP services are restored.

Google Cloud outage

Thu, 12 Jun 2025 19:23:00 -0000

Services are beginning to come back online - play.instruqt.com and docs.instruqt.com are accessible. Tracks will not play at this time while GCP service issues continue to resolve.

Google Cloud outage

Thu, 12 Jun 2025 17:57:00 -0000

We are aware of the ongoing widespread outage with Google Cloud Platform (GCP), which is currently impacting Instruqt services. Our team is actively monitoring the situation.

Increased Latency in VM Startups

Tue, 20 May 2025 12:30:00 -0000

Google Cloud Platform (GCP) has resolved the incident impacting VM provisioning. We have observed recovery in our internal metrics, and track start-up times have returned to normal. The issue is now considered resolved on our side as well. Thank you for your patience while we monitored and mitigated the impact.

Increased Latency in VM Startups

Tue, 20 May 2025 11:15:00 -0000

We are currently observing increased latency in VM provisioning times, which appears to be related to an ongoing issue with Google Cloud Platform (GCP). We are actively monitoring the situation and its impact on track start-ups. For critical workloads, we recommend using hot start pools as a mitigation strategy.

Maintenance: Database Maintenance

Wed, 30 Apr 2025 12:20:42 -0000

Scheduled Maintenance Notice We will be performing routine maintenance on May 4, 2025, starting at 02:00:00 (UTC). The maintenance is expected to last approximately 1 hour. During this time, you may experience brief errors or interruptions in service. We apologize for any inconvenience and appreciate your patience as we continue to improve our services.

AWS account provisioning

Tue, 29 Apr 2025 07:00:00 -0000

We have reallocated older unused accounts into the general pool which has freed up significant amount capacity. We are comfortable with the amount of capacity that we now have and are thus closing this incident. We continue to work with AWS to extend this quota further for future use. We are also closely monitoring the amount of available AWS accounts that we can allocate.

AWS account provisioning

Fri, 18 Apr 2025 13:00:00 -0000

We have freed up more AWS accounts in our pool to further mitigate the immediate issue. We're still working with AWS to increase our total capacity.

AWS account provisioning

Wed, 16 Apr 2025 14:42:00 -0000

Provisioning extra AWS accounts is currently degraded. We maintain a pool of AWS account that are ready to use. We have reached our AWS Account quota and thus cannot currently add more accounts to the per customer pool. This will only affect customers that are starting tracks that contain AWS Cloud Accounts to a volume that has not yet been tested for that customer before. Running tracks that can use an existing AWS account are not affected. We have reached out to AWS to extend our Account quantity quota.

Unable to provision VMs and containers in us-central1 region

Thu, 10 Apr 2025 19:14:00 -0000

The issue affecting sandbox creation with VMs or containers in the us-central1 region has been resolved. All services are now operating normally.

Unable to provision VMs and containers in us-central1 region

Thu, 10 Apr 2025 18:23:00 -0000

A fix has been implemented, and we're actively monitoring the situation to confirm that the issue is fully resolved.

Unable to provision VMs and containers in us-central1 region

Thu, 10 Apr 2025 17:45:00 -0000

We're currently investigating an issue affecting the creation of sandboxes using VMs or containers in the us-central1 region. In the meantime, for the hot start pools, we recommend creating in the europe-west1 region to avoid disruption. Our team is actively investigating the issue and working to resolve it as quickly as possible. We appreciate your patience and will provide updates here as they become available.

Maintenance: Planned Maintenance

Tue, 01 Apr 2025 09:08:01 -0000

Our CRM integration platform provider will be performing scheduled maintenance on Saturday, April 5, 2025, from 11:00 PM to 12:00 AM PDT (UTC-7). During this time, CRM integration-related services (Salesforce & HubSpot) may be temporarily unavailable. Any triggered events will be cached and processed once maintenance is complete. We expect no impact on end-user experience.

Maintenance: Scheduled Maintenance

Thu, 20 Mar 2025 14:57:16 -0000

Scheduled Maintenance Notice We will be performing routine maintenance on March 23, 2025, starting at 01:00 CET. The maintenance is expected to last approximately 1 hour. During this time, you may experience brief interruptions in the following features: 🔹 Tracks with version control enabled: Viewing and saving track configurations 🔹 Live events: • Sending and receiving chat messages • Raising and lowering hand • Screen sharing We apologize for any inconvenience and appreciate your patience as we continue to improve our services.

Google Cloud Accounts in Sandboxes Unavailable

Mon, 13 Jan 2025 17:46:00 -0000

Billing activation times for GCP project have recovered to normal. We will continue to closely monitor the situation.

Google Cloud Accounts in Sandboxes Unavailable

Mon, 13 Jan 2025 16:20:00 -0000

We have identified prolonged activation times for billing, which has impacted the usability of new Google cloud accounts. Our automated tests indicate that the issue is gradually recovering. As a temporary workaround, we recommend implementing a 10-minute delay before initiating resource creation on Google Cloud accounts for new sandboxes. We are closely monitoring the situation through automated tests and will provide further updates once we confirm that Google’s billing functionality has returned to normal. A support ticket has been submitted to Google, and we are actively tracking its status.

Google Cloud Accounts in Sandboxes Unavailable

Mon, 13 Jan 2025 15:01:00 -0000

We are currently investigating an issue where Google cloud accounts added to sandboxes are unusable due to a billing-related problem. Our team is actively working to identify the root cause and resolve this as quickly as possible. Thank you for your patience, and we will provide updates as soon as more information is available.

Maintenance: Database Maintainence window

Wed, 11 Dec 2024 08:28:52 -0000

Maintenance Successfully Completed We are pleased to inform you that the scheduled maintenance has been successfully completed. All systems are fully operational, and the website is now running smoothly. Thank you for your patience and understanding. If you experience any issues, please don't hesitate to contact our support team.

Content Creator UI is currently unavailable for some tracks

Wed, 04 Dec 2024 17:26:00 -0000

The fix has been applied and live. The Content Creator UI is up and responding again. We will continue to monitor for any impacts.

Content Creator UI is currently unavailable for some tracks

Wed, 04 Dec 2024 17:18:00 -0000

A fix has been generated and is being applied. We hope to restore services soon.

Content Creator UI is currently unavailable for some tracks

Wed, 04 Dec 2024 16:00:00 -0000

The Content Creator UI is currently unavailable on tracks with git integration enabled. We have isolated the issue and are working to resolve it. No track configuration is impacted, and plays of the tracks are not impacted.

High database load

Wed, 20 Nov 2024 13:24:00 -0000

We have made some small changes to our chat tooling to ensure our database load remains within accepted bounds. Also we are confirming that we did not experience any unforeseen customer impact during this outage.

High database load

Wed, 20 Nov 2024 11:40:00 -0000

The high database load has subsided. We are continuing to closely monitor and do not expect any degraded service. Next update we will likely post a resolution.

High database load

Wed, 20 Nov 2024 10:30:00 -0000

We have isolated the issue and we are currently rolling out changes to reduce the load. We are continuing to monitor the situation closely.

High database load

Wed, 20 Nov 2024 09:15:00 -0000

We are currently experiencing high database load. We do not expect this to be causing impact to existing plays or tracks. We are monitoring the situation closely.

Degraded Performance

Fri, 15 Nov 2024 16:55:00 -0000

✅ The issue affecting our platform has been fully resolved. The cause was identified and fixed. We appreciate your patience and apologize for any inconvenience caused.

Degraded Performance

Fri, 15 Nov 2024 16:26:00 -0000

We have mitigated the issue. We are currently monitoring the status and investigating the root cause. Thanks for your patience!

Degraded Performance

Fri, 15 Nov 2024 15:55:00 -0000

We are currently experiencing degraded performance across some parts of our site. Our team is actively investigating the issue and working to resolve it as quickly as possible. We appreciate your patience and will provide updates here as they become available.

Maintenance: Planned Maintenance

Fri, 08 Nov 2024 10:35:16 -0000

We will be performing scheduled maintenance on 10 November 2024 from 10:00 to 11:00 (CET). During this time, tracks including Container or Website service (Virtual Browser) hosts may experience intermittent disruptions, including temporary slowness or brief interruptions. Other services will remain unaffected. We apologize for any inconvenience and appreciate your patience as we work to improve our systems. Thank you for your understanding.

Degraded performance

Thu, 31 Oct 2024 23:00:00 -0000

✅ The issue affecting our platform has been fully resolved. The cause was identified and fixed. We appreciate your patience and apologize for any inconvenience caused.

Degraded performance

Thu, 31 Oct 2024 20:43:00 -0000

We have identified the cause of the degraded performance being a spike in load of our database. We have identified the root cause and are in the process of resolving the issue.

Degraded performance

Thu, 31 Oct 2024 19:10:00 -0000

Maintenance: Scheduled Maintenance

Thu, 31 Oct 2024 17:21:38 -0000

We will be performing routine maintenance on 03.11.2024 starting at 09:00 CET. The expected duration is 2 hours. In this duration, there might be brief connection interruptions to the sandbox hosts, including: - Terminal - Service / Web Application - Virtual Browser / Web Page - Code Editor Thank you for your patience as we work to improve our services.

Disabled reporting queries on play.instruqt.com

Wed, 02 Oct 2024 10:35:00 -0000

We have identified the root cause, and have released a fix to restore reporting functionality. We'll continue to closely monitor the situation.

Disabled reporting queries on play.instruqt.com

Tue, 01 Oct 2024 15:50:00 -0000

We have identified the cause of the degraded performance, being our reporting functionality. To mitigate the issue, we have had to disable our reporting query functionality until further notice. We're working on steps to remediate the disabled functionality as soon as possible. There is no data loss, all reporting data is still being recorded, we have only disabled the reporting views.

Disabled reporting queries on play.instruqt.com

Tue, 01 Oct 2024 14:35:00 -0000

We're seeing degraded performance for play.instruqt.com, due to increased database load. We're investigating the root cause.

Unable to provision per sandbox SSL certificates

Wed, 31 Jan 2024 20:05:00 -0000

Service has been restored.

Unable to provision per sandbox SSL certificates

Wed, 31 Jan 2024 18:51:00 -0000

While we are seeing still elevated error rates, some certificates are being provisioned successfully. When unable to start a sandbox environment due to certificate errors, try restarting it, as it might succeed the second time around. We'll keep monitoring the situation.

Unable to provision per sandbox SSL certificates

Wed, 31 Jan 2024 16:52:00 -0000

We're currently seeing errors from our SSL provider Let's Encrypt, rendering us unable to provision new certificates. This affects all VMs that have the "Provision SSL Certficate" option enabled. We'll keep monitoring the situation, and update here when we have more information.

GCP is having issues creating new projects

Tue, 21 Nov 2023 09:37:00 -0000

The issues with the GCP Projects API seem to be resolved, and our GCP projects are working again as expected.

GCP is having issues creating new projects

Tue, 21 Nov 2023 06:41:00 -0000

GCP is having issues with their projects API, not allowing us to create new projects. This affects new plays using our GCP projects. Running instances are not affected.

Unable to start tracks due to invalid credentials

Fri, 29 Sep 2023 06:44:00 -0000

We've identified an issue where GCP has problems issuing new credentials. We've should have mitigated these issues. We'll keep monitoring the situation.

Unable to start tracks due to invalid credentials

Fri, 29 Sep 2023 01:58:00 -0000

We're experiencing issues with provisioning sandbox environments, due to invalid credentials for an internal API.