Automating TFS to Jira: Preserving Test Hierarchy and Data Integrity with Python

Migrations are rarely clean. When moving from legacy on-premise Team Foundation Server (TFS) or Azure DevOps (ADO) to Jira, the data transfer is the easy part. The real challenge is context. Most out-of-the-box export tools flatten your data, turning a carefully organized tree of Test Plans and Suites into a chaotic list of disconnected tickets.

For an enterprise application, losing that hierarchy means losing years of organizational knowledge. At Deviqon Labs, we prioritize data integrity. When faced with a migration involving thousands of tasks and test cases nested in deep folder structures, we didn't resort to manual reorganization. We engineered two distinct Python solutions.

The Two-Pronged Approach

A "one-size-fits-all" script rarely works because Tasks and Test Cases in ADO behave differently. Tasks are flat; they live in a backlog. Test Cases live in a hierarchy of Plans and Suites. To handle this, we split our logic:

1. For Tasks & Bugs: A fast WIQL (Work Item Query Language) extractor.
2. For Test Cases: A recursive folder crawler that maps the tree structure.

Part 1: The Tasks (WIQL Extraction)

For standard work items, speed is key. We utilized ADO’s WIQL endpoint to fetch IDs in bulk, then iterated through them to download attachments and history. The critical component here is handling the rate limiting and timeouts common with older on-premise servers.

def get_all_work_item_ids():
    # WIQL allows us to grab thousands of IDs efficiently
    query = {"query": "Select [System.Id] From WorkItems Where [System.TeamProject] = @project Order By [System.Id] Desc"}
    try:
        r = requests.post(f"{ADO_BASE_URL}/_apis/wit/wiql?api-version=6.0", json=query, headers=headers)
        r.raise_for_status()
        return [item['id'] for item in r.json()['workItems']]
    except Exception as e:
        logger.critical(f"Connection Failed: {e}")
        exit()

This script generates a flat list of JSON files, ready for mapping to Jira Issue Types.

Part 2: The Test Plans (Recursive Crawling)

This is where standard tools fail. We needed to replicate the folder structure on our local machine exactly as it appeared in TFS. This required a recursive function to walk the Test Plan tree.

1. Solving the "On-Prem" Connectivity Issue

Legacy TFS servers often have resolution issues (e.g., internal DNS failures or port blocking). Before attempting to crawl thousands of items, our script runs a diagnostic check using Python's socket library. This prevents the script from crashing mid-execution due to a blip in the VPN or internal network.

def check_connection():
    parsed = urlparse(ADO_BASE_URL)
    hostname = parsed.hostname
    port = parsed.port or 80

    print(f"--- DIAGNOSTIC: Checking connection to '{hostname}' ---")
    try:
        # verify DNS resolution
        ip = socket.gethostbyname(hostname)
        print(f"[OK] DNS Resolved '{hostname}' to {ip}")

        # verify TCP socket connection
        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        sock.settimeout(5)
        result = sock.connect_ex((hostname, port))
        if result == 0:
            return True
    except Exception as e:
        print(f"[FATAL] Connection check error: {e}")
        return False

Handling this level of environmental instability is a common challenge in the enterprise modernization projects our teams at Deviqon Labs focus on.

2. The Recursive Mapper

We fetch a Plan, then find its Suites. If a Suite contains children, the function calls itself, building a directory path string (e.g., /Plan A/Regression/Payment Module).

def get_suites_for_plan(plan_id, plan_name):
    url = f"{ADO_BASE_URL}/_apis/test/plans/{plan_id}/suites?api-version=6.0"
    resp = safe_api_request(url)
    suites = resp.json()['value']

    # Map Children to Parents
    children_map = {}
    root_suite = None

    for s in suites:
        if 'parentSuite' in s:
            pid = s['parentSuite']['id']
            children_map.setdefault(pid, []).append(s)
        else:
            root_suite = s

    final_paths = [] 

    # Recursive path builder
    def traverse(current_suite, current_path):
        s_name = clean_filename(current_suite['name'])
        full_path = os.path.join(current_path, s_name)

        final_paths.append((current_suite['id'], full_path))

        children = children_map.get(current_suite['id'], [])
        for child in children:
            traverse(child, full_path)

    if root_suite:
        traverse(root_suite, "")

    return final_paths

3. Parsing XML Steps

Finally, ADO stores test steps in a verbose XML format (`Microsoft.VSTS.TCM.Steps`). We strip the HTML tags and structure the data into clean JSON arrays for the Jira description field.

def parse_ado_steps(xml_string):
    if not xml_string: return []
    steps_data = []
    try:
        # Wrap in root to handle fragment errors
        root = ET.fromstring(f"{xml_string}")
        for i, step in enumerate(root.findall('.//step'), 1):
            params = step.findall('parameterizedString')
            # Custom clean_html regex function applied here
            action = clean_html(params[0].text)
            expected = clean_html(params[1].text)
            steps_data.append({"index": i, "action": action, "expected": expected})
    except:
        return [{"index": 1, "action": "XML Error", "expected_result": "Could not parse"}]
    return steps_data

The Result

By running these scripts, we generate a local "Mirror" of the TFS instance. Tasks are flattened for the backlog, while Test Cases sit in folders mirroring their exact Test Plan hierarchy. This allows us to validate assets, check attachment sizes, and clean data before we ever touch the Jira API.

Bridging the Gap to Jira

Once we have this clean local mirror, the upload logic is straightforward. We map the folder paths to Jira Components (or Xray Test Repositories), ensuring that when a Test Case is created in Jira, it automatically lands in the correct organizational bucket. This transforms a blind data dump into a structured import.

From Theory to Practice

The concepts in this article, robust error handling, recursive crawling, and protocol diagnostics, are foundational for maintaining data integrity during system migrations. If you are exploring how these techniques apply to your own legacy systems, or if you are facing a unique technical hurdle, our senior specialists are available for a technical discussion.

Contact Deviqon Labs to discuss your project.