Multiple versions of the connector

Oct 13, 2025 by pinkandtrash.com 53 views

Introduction

Airbyte is a popular open-source data integration platform that allows users to integrate various data sources and destinations. However, sometimes users may encounter errors while setting up their connectors. In this article, we will discuss the issue of "Failed to fetch source definitions" when using the MSSQL-V2 connector.

Error Details

The error message is as follows:

$`Internal Server Error: Duplicate key 37a928c1-2d5c-431a-a97d-ae236bd1ea0c (attempted merging values io.airbyte.config.StandardDestinationDefinition@2d65f7f0[destinationDefinitionId=37a928c1-2d5c-431a-a97d-ae236bd1ea0c,name=MS SQL Server V2,dockerRepository=airbyte/destination-mssql-v2,dockerImageTag=0.1.15,documentationUrl=https://docs.airbyte.com/integrations/destinations/mssql-v2,icon=icon.svg,spec=io.airbyte.protocol.models.ConnectorSpecification@6f4a3078[documentationUrl=https://docs.airbyte.com/integrations/destinations/mssql-v2,changelogUrl=<null>,connectionSpecification={"$schema":"http://json-schema.org/draft-07/schema#","title":"MSSQL V2 Destination Specification","type":"object","additionalProperties":true,"properties":{"host":{"type":"string","description":"The host name of the MSSQL database.","title":"Host","order":0},"port":{"type":"integer","description":"The port of the MSSQL database.","title":"Port","minimum":0,"maximum":65536,"examples":["1433"],"order":1},"database":{"type":"string","description":"The name of the MSSQL database.","title":"DB Name","order":2},"schema":{"type":"string","description":"The default schema tables are written to if the source does not specify a namespace. The usual value for this field is \"public\".","title":"Default Schema","examples":["public"],"default":"public","order":3},"user":{"type":"string","description":"The username which is used to access the database.","title":"User","order":4},"password":{"type":"string","description":"The password associated with this username.","title":"Password","airbyte_secret":true,"order":5},"jdbc_url_params":{"type":"string","description":"Additional properties to pass to the JDBC URL string when connecting to the database formatted as 'key=value' pairs separated by the symbol '&'. (example: key1=value1&key2=value2&key3=value3).","title":"JDBC URL Params","order":6},"ssl_method":{"oneOf":[{"title":"Unencrypted","type":"object","additionalProperties":true,"description":"The data transfer will not be encrypted.","properties":{"name":{"type":"string","enum":["unencrypted"],"default":"unencrypted"}},"required":["name"]},{"title":"Encrypted (trust server certificate)","type":"object","additionalProperties":true,"description":"Use the certificate provided by the server without verification. (For testing purposes only!)","properties":{"name":{"type":"string","enum":["encrypted_trust_server_certificate"],"default":"encrypted_trust_server_certificate"}},"required":["name"]},{"title":"Encrypted (verify certificate)","type":"object","additionalProperties":true,"description":"Verify and use the certificate provided by the server.","properties":{"name":{"type":"string","enum":["encrypted_verify_certificate"],"default":"encrypted_verify_certificate"},"trustStoreName":{"type":"string","description":"Specifies the name of the trust store.","title":"Trust Store Name","order":1},"trustStorePassword":{"type":"string","description":"Specifies the password of the trust store.","title":"Trust Store Password","airbyte_secret":true,"order":2},"hostNameInCertificate":{"type":"string","description":"Specifies the host name of the server. The value of this property must match the subject property of the certificate.","title":"Host Name In Certificate","order":3}},"required":["name"]}],"description":"The encryption method which is used to communicate with the database.","title":"SSL Method","order":7,"type":"object"},"load_type":{"oneOf":[{"title":"Insert Load","type":"object","additionalProperties":true,"description":"Configuration details for using the INSERT loading mechanism.","properties":{"load_type":{"type":"string","enum":["INSERT"],"default":"INSERT"}},"required":["load_type"]},{"title":"Bulk Load","type":"object","additionalProperties":true,"description":"Configuration details for using the BULK loading mechanism.","properties":{"load_type":{"type":"string","enum":["BULK"],"default":"BULK"},"azure_blob_storage_account_name":{"type":"string","description":"The name of the Azure Blob Storage account. See: https://learn.microsoft.com/azure/storage/blobs/storage-blobs-introduction#storage-accounts","title":"Azure Blob Storage Account Name","examples":["mystorageaccount"]},"azure_blob_storage_container_name":{"type":"string","description":"The name of the Azure Blob Storage container. See: https://learn.microsoft.com/azure/storage/blobs/storage-blobs-introduction#containers","title":"Azure Blob Storage Container Name","examples":["mycontainer"]},"shared_access_signature":{"type":"string","description":"A shared access signature (SAS) provides secure delegated access to resources in your storage account. See: https://learn.microsoft.com/azure/storage/common/storage-sas-overview","title":"Shared Access Signature","examples":["a012345678910ABCDEFGH/AbCdEfGhEXAMPLEKEY"],"airbyte_secret":true,"always_show":true},"bulk_load_data_source":{"type":"string","description":"Specifies the external data source name configured in MSSQL, which references the Azure Blob container. See: https://learn.microsoft.com/sql/t-sql/statements/bulk-insert-transact-sql","title":"BULK Load Data Source","examples":["MyAzureBlobStorage"],"order":4,"always_show":true},"bulk_load_validate_values_pre_load":{"type":"boolean","description":"When enabled, Airbyte will validate all values before loading them into the destination table. This provides stronger data integrity guarantees but may significantly impact performance.","title":"Pre-Load Value Validation","examples":["false"],"default":false,"order":5,"always_show":false}},"required":["load_type","azure_blob_storage_account_name","azure_blob_storage_container_name","shared_access_signature","bulk_load_data_source"]}],"description":"Specifies the type of load mechanism (e.g., BULK, INSERT) and its associated configuration.","title":"Load Type","always_show":true,"order":8,"type":"object"}},"required":["host","port","database","schema","user","ssl_method","load_type"]},supportsIncremental=true,supportsNormalization=false,supportsDBT=false,supportedDestinationSyncModes=[overwrite, append, append_dedup],authSpecification=<null>,advancedAuth=<null>,protocolVersion=<null>,additionalProperties={}],tombstone=false,_public=true,custom=false,releaseStage=alpha,releaseDate=<null>,resourceRequirements=<null>,protocolVersion=0.2.0,normalizationConfig=<null>,supportsDbt=<null>,allowedHosts=<null>,additionalProperties={connectorBuildOptions={baseImage=docker.io/airbyte/java-connector-base:2.0.0@sha256:5a1a21c75c5e1282606de9fa539ba136520abe2fbd013058e988bb0297a9f454}, connectorTestSuitesOptions=[{suite=unitTests, testSecrets=null, testConnections=null}, {suite=integrationTests, testSecrets=[{name=SECRET_DESTINATION_MSSQL_V2_BULK_LOAD_CONFIG, fileName=bulk_upload_config.json, secretStore={alias=airbyte-connector-testing-secret-store, type=GSM}}], testConnections=null}], language=java, supportLevel=community, tags=[language:java], supportsRefreshes=true, ab_internal={sl=100, ql=100}, license=ELv2, generated={git={commit_sha=600587baa92fc46ef090e2a019ca105ca4649fd5, commit_timestamp=2025-03-10T08:35:24-07:00, commit_author=Johnny Schmidt, commit_author_email=john.schmidt@airbyte.io}, source_file_info={metadata_etag=CJrH2bXw/4sDEAE=, metadata_file_path=metadata/airbyte/destination-mssql-v2/latest/metadata.yaml, metadata_bucket_name=prod-airbyte-cloud-connector-metadata-service, metadata_last_modified=2025-03-10T15:57:16.158000+00:00, registry_entry_generated_at=2025-03-10T16:00:36.748103}, metrics={all={connector_type=destination, connector_definition_id=37a928c1-2d5c-431a-a97d-ae236bd1ea0c, connector_name=MS SQL Server V2, docker_repository=airbyte/destination-mssql-v2, airbyte_platform=all, connector_version=all, sync_success_rate=medium, usage=high}, cloud={connector_type=destination, connector_definition_id=37a928c1-2d5c-431a-a97d-ae236bd1ea0c, connector_name=MS SQL Server V2, docker_repository=airbyte/destination-mssql-v2, airbyte_platform=cloud, connector_version=0.1.14, sync_success_rate=medium, usage=high}, oss={connector_type=destination, connector_definition_id=37a928c1-2d5c-431a-a97d-ae236bd1ea0c, connector<br/> **MSSQL-V2] Failed to fetch source definitions**$

Q&A

Q: What is the cause of the "Failed to fetch source definitions" error when using the MSSQL-V2 connector? A: The error is caused by a duplicate key in the connector's metadata. This can happen when there are multiple versions of the connector installed on the system, or when the connector's metadata is not properly updated.

Q: How can I resolve the "Failed to fetch source definitions" error? A: To resolve the error, you can try the following:

Update the connector: Make sure you are using the latest version of the MSSQL-V2 connector. You can check for updates in the Airbyte dashboard or by visiting the Airbyte website.
Delete the connector's metadata: Delete the connector's metadata file to force the system to recreate it. You can do this by deleting the metadata directory in the Airbyte configuration directory.
Check for duplicate keys: Check the connector's metadata for duplicate keys. If you find any, remove them and try again.

Q: What are the possible causes of duplicate keys in the connector's metadata? A: The possible causes of duplicate keys in the connector's metadata are:

Multiple versions of the connector: If you have multiple versions of the connector installed on the system, it can cause duplicate keys in the metadata.
Connector metadata not properly updated: If the connector's metadata is not properly updated, it can cause duplicate keys.
Corrupted metadata file: If the metadata file is corrupted, it can cause duplicate keys.

Q: How can I prevent duplicate keys in the connector's metadata? A: To prevent duplicate keys in the connector's metadata, you can:

Use a single version of the connector: Use a single version of the connector to avoid multiple versions being installed on the system.
Update the connector regularly: Update the connector regularly to ensure that the metadata is properly updated.
Check the metadata file regularly: Check the metadata file regularly to ensure that it is not corrupted.

Q: What are the consequences of having duplicate keys in the connector's metadata? A: The consequences of having duplicate keys in the connector's metadata are:

Error messages: You will receive error messages when trying to use the connector.
Connector not working: The connector may not work properly or at all.
Data loss: In severe cases, data loss may occur.

Q: How can I troubleshoot the "Failed to fetch source definitions" error? A: To troubleshoot the error, you can:

Check the Airbyte logs: Check the Airbyte logs for any error messages related to the connector.
Check the connector's metadata: Check the connector's metadata for any issues.
Contact Airbyte support: Contact Airbyte support for further assistance.

Conclusion

The "Failed to fetch source definitions" error when using the MSSQL-V2 connector can be caused by a duplicate key in the connector's metadata. To resolve the error, you can try updating the connector, deleting the connector's metadata, or checking for duplicate keys. To prevent duplicate keys, you can use a single version of the connector, update the connector regularly, and check the metadata file regularly. If you are experiencing issues with the connector, you can troubleshoot the error by checking the Airbyte logs, checking the connector's metadata, or contacting Airbyte support.