Here we provide supplementary data supporting RQ1 results.
Requirements Quality Results:
Initial Submissions: Table below shows the average score for each requirement attribute for initial submissions (denoted by H) in processes A+ and A-.
| Req. Quality Metric | A+ | A- |
|---|---|---|
| Complete | 3.1 | 3.3 |
| Consistent | 3.5 | 3.3 |
| Unambiguous | 3.5 | 3.3 |
| Focused | 3.4 | 3.3 |
| Relevant | 3.2 | 3.1 |
| Feasible | 4.8 | 5.0 |
| Verifiable/Measurable | 4.8 | 4.1 |
| Correctly classified | 4.7 | 4.2 |
| Well-formatted | 3.9 | 2.6 |
| Total Quality (H) | 77% | 72% |
ChatGPT’s Outputs: Table below shows the average score for each requirement attribute for ChatGPT’s outputs (denoted by G) in processes A+, A-, and B.
| Req. Quality Metric | A+ | A- | B |
|---|---|---|---|
| Complete | 4.1 | 4.6 | 4.2 |
| Consistent | 2.6 | 1.7 | 3.2 |
| Unambiguous | 2.9 | 2.9 | 3.6 |
| Focused | 3.9 | 3.7 | 3.6 |
| Relevant | 3.2 | 3 | 2.8 |
| Feasible | 2.7 | 2.3 | 2.3 |
| Verifiable/Measurable | 2.9 | 1.4 | 3.0 |
| Correctly classified | 4.6 | 4.43 | 4.5 |
| Well-formatted | 1.6 | 0.3 | 3.5 |
| Total Quality (G) | 63% | 54% | 68% |
Final Submissions: Table below shows the average score for each requirement attribute for final submissions (denoted by F) in processes A+, A-, and B.
| Req. Quality Metric | A+ | A- | B |
|---|---|---|---|
| Complete | 4.7 | 4.7 | 4.6 |
| Consistent | 4.2 | 3.2 | 4.3 |
| Unambiguous | 3.5 | 3.1 | 3.3 |
| Focused | 4.2 | 3.7 | 4.1 |
| Relevant | 3.3 | 3 | 2.8 |
| Feasible | 4.5 | 4.14 | 3.5 |
| Verifiable/Measurable | 4.5 | 3.8 | 3.6 |
| Correctly classified | 4.6 | 4.5 | 4.5 |
| Well-formatted | 3.6 | 3.5 | 3.8 |
| Total Quality (F) | 82% | 75% | 77% |
Prompt Quality Results:
Table below shows the average score for each prompt quality metric in processes A+, A-, and B.
| Prompt Quality Metric | A+ | A- | B |
|---|---|---|---|
| Course Setup | 1.1 | 0.6 | 1.9 |
| Project Setup | 2.8 | 1.0 | 3.0 |
| Explicit Requests | 3.8 | 3.3 | 3.9 |
| Expected Content | 3.0 | 2.3 | 4.4 |
| Expected Format | 1.9 | 0.9 | 3.3 |
| Personas | 0.5 | 0.3 | 0.5 |
| Examples | 0.2 | 0 | 0.3 |
| Avg. “How to ask” | 1.9 | 1.2 | 2.4 |
| What to ask | 4 | 3.6 | 3.9 |
| Total Quality | 30% | 17% | 38% |