Azure Service Bus queue state change
Last updated on
Azure Service Bus queue state change modifies the status of Azure Service Bus queues for a specific chaos duration.
- It validates the resilience of applications that depend on Azure Service Bus messaging by testing how they handle queue disruptions.
- The fault can change queue statuses to disabled, send-disabled, or receive-disabled states.
Use cases
Azure Service Bus queue state change:
- Determines the resilience of applications when Service Bus queues become unavailable or have restricted operations.
- Validates that message processing systems properly handle queue disruptions and implement appropriate retry mechanisms.
- Tests the behavior of distributed systems when message queues are disabled, ensuring graceful degradation.
- Verifies monitoring and alerting systems properly detect Service Bus queue state changes.
Prerequisites
- Kubernetes >= 1.17
- Appropriate Azure access to manage Service Bus queues (read and write permissions on Service Bus namespace and queues).
- Azure Service Bus queues should be in an Active state before chaos injection.
- Use Azure file-based authentication to connect to Azure using the Azure Go SDK. To generate the auth file, run
az ad sp create-for-rbac --sdk-auth > azure.authAzure CLI command. - Kubernetes secret should contain the auth file created in the previous step in the
CHAOS_NAMESPACE. Below is a sample secret file:
apiVersion: v1
kind: Secret
metadata:
name: cloud-secret
type: Opaque
stringData:
azure.auth: |-
{
"clientId": "XXXXXXXXX",
"clientSecret": "XXXXXXXXX",
"subscriptionId": "XXXXXXXXX",
"tenantId": "XXXXXXXXX",
"activeDirectoryEndpointUrl": "XXXXXXXXX",
"resourceManagerEndpointUrl": "XXXXXXXXX",
"activeDirectoryGraphResourceId": "XXXXXXXXX",
"sqlManagementEndpointUrl": "XXXXXXXXX",
"galleryEndpointUrl": "XXXXXXXXX",
"managementEndpointUrl": "XXXXXXXXX"
}
tip
If you change the secret key name from azure.auth to a new name, ensure that you update the AZURE_AUTH_LOCATION environment variable in the chaos experiment with the new name.
Required Azure permissions
The service principal needs the following permissions:
- Azure Service Bus Data Owner or Azure Service Bus Data Contributor role on the target namespace
- Or custom role with these permissions:
Microsoft.ServiceBus/namespaces/queues/readMicrosoft.ServiceBus/namespaces/queues/write
Mandatory tunables
| Tunable | Description | Notes |
|---|---|---|
| RESOURCE_GROUP | Name of the Azure resource group where the Service Bus namespace exists. | For example, rg-azure-servicebus. For more information, go to resource group field in the YAML file. |
| AZURE_NAMESPACE_NAME | Name of the Azure Service Bus namespace. | For example, my-servicebus-namespace. For more information, go to Azure namespace name. |
| QUEUE_NAMES | Comma-separated list of Service Bus queue names to target. | For example, queue-1,queue-2,queue-3. For more information, go to queue names. |
| ACTION | Target status to set for the queue(s). | Supported values: disabled, senddisabled, receivedisabled. Defaults to disabled. For more information, go to action. |
Optional tunables
| Tunable | Description | Notes |
|---|---|---|
| TOTAL_CHAOS_DURATION | Duration that you specify, through which chaos is injected into the target resource (in seconds). | Defaults to 30s. For more information, go to duration of the chaos. |
| CHAOS_INTERVAL | Time interval between successive chaos iterations (in seconds). | Defaults to 30s. For more information, go to chaos interval. |
| SEQUENCE | Sequence of chaos execution for multiple queues. | Defaults to parallel. Also supports serial sequence. For more information, go to sequence of chaos execution. |
| RAMP_TIME | Period to wait before and after injecting chaos (in seconds). | For example, 30s. For more information, go to ramp time. |
| DEFAULT_HEALTH_CHECK | Determines if you wish to run the default health check which is present inside the fault. | Default: 'true'. Validates queues are in Active state before chaos. For more information, go to default health check. |
Change queue state by name
It specifies a comma-separated list of Service Bus queue names subject to state change. Tune it by using the QUEUE_NAMES, AZURE_NAMESPACE_NAME, RESOURCE_GROUP, and ACTION environment variables.
Use the following example to tune it:
# change state of Azure Service Bus queues for a certain chaos duration
apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
name: engine-nginx
spec:
engineState: "active"
annotationCheck: "false"
chaosServiceAccount: litmus-admin
experiments:
- name: azure-service-bus-queue-state-change
spec:
components:
env:
# comma-separated names of the Azure Service Bus queues
- name: QUEUE_NAMES
value: 'queue-1,queue-2'
# name of the Azure Service Bus namespace
- name: AZURE_NAMESPACE_NAME
value: 'my-servicebus-namespace'
# name of the resource group
- name: RESOURCE_GROUP
value: 'rg-azure-servicebus'
# target state for the queues
- name: ACTION
value: 'disabled'
- name: TOTAL_CHAOS_DURATION
value: '60'