Testing Agile

Choosing the right Test Case Management System (TCMS) can be a journey filled with technical challenges, especially when working with open-source tools or tools that require a considerable amount of setup. Here’s a candid look at my experiences setting up Kiwi, Squash, and TestLink, and the struggles I faced along the way.

Kiwi TCMS

My attempt to set up Kiwi TCMS started with high hopes, but it quickly turned into a full-day endeavor. The most challenging part was initializing the database, which was required for Kiwi to function. Despite following the setup documentation and troubleshooting guides, I ran into several errors related to database configuration. After hours of working through the setup, I tried one more time and I finally succeeded, however now I am stuck at login issues. Ultimately I decided that Kiwi’s setup process was more complex and time-consuming than I had anticipated, and I didn’t want to spend any more time struggling with the configurations. This experience highlighted the need for a TCMS with a more straightforward setup, especially when working within limited time constraints.

Squash TM

Squash TM was another open-source tool that appeared promising at first glance. However, the setup process brought its own set of challenges. Squash offers multiple Docker images, such as the signed image squashtest/squash and the unsigned squashtest/squash-tm, which is being deprecated soon. I initially downloaded the unsigned image without realizing it was outdated, which added a layer of confusion. Moreover, I needed to understand which images were required to get Squash fully functional. While Squash has a lot of potential, especially for teams with tight budgets, the setup experience made it clear that it requires significant technical knowledge. I ended up working with chat gpt for troubleshooting although things seemed fine I was still not able to access the localhost. If your team doesn’t have extensive experience with Docker or is looking for a simpler setup process, Squash TM might not be the most accessible option.

TestLink

TestLink, another open-source TCMS, offered a bit of a different experience. I had used TestLink in the past, so I was familiar with its basic features. However, I encountered several compatibility issues during setup. The newer XAMPP versions came with a PHP version unsupported by TestLink, so I downgraded XAMPP to make it compatible. Despite this, TestLink still didn’t recognize the necessary SQL extension, which created additional setup headaches. This experience highlights the challenges of working with open-source tools, where dependency issues can often slow down the process. After navigating through this setup , I am quite convinced that Test Link is no longer supported or maintained and that is a risk with using open-source tools.

Lessons Learned

These experiences underscored the importance of selecting a TCMS that matches not only your functional requirements but also your technical capabilities and resource availability. Here are a few key takeaways from my experience with Kiwi, Squash, and TestLink:

1. Evaluate Technical Complexity: Tools like Kiwi and Squash require a level of technical expertise for setup that may not be feasible for every team. If you’re not familiar with Docker or database configuration, the setup can quickly become overwhelming.
2. Check Documentation and Support: While all three tools have documentation, I found it to be inconsistent and sometimes outdated, especially with TestLink. Before committing to a tool, it’s worth spending some time reviewing the documentation and seeing if there’s an active community or support channel.
3. Assess Feature Requirements Early On: Although these tools are attractive because they’re budget-friendly, it’s essential to ensure they offer the features you truly need. I realized I needed a TCMS that could handle image attachments and more modern integrations, which these tools didn’t fully support.
4. Plan for Long-Term Maintenance: With open-source tools, you may encounter issues with software updates, deprecated images, or limited support, which can create additional maintenance overhead. Consider the long-term implications of maintaining the tool, especially if your team has limited resources.

I finally gave up and made myself a trial account on TestRail, although I know it’s outside of my budget, at least I can focus on my work for a month and decide later what to do next.

My journey with Kiwi, Squash, and TestLink reinforced the importance of finding a TCMS that not only offers the necessary features but is also easy to set up and maintain. While these tools are free and supposedly functional, they come with a learning curve and require a certain level of technical commitment. Ultimately, choosing the right TCMS involves balancing functionality, ease of setup, and the resources your team can dedicate to getting—and keeping—the tool up and running.