"Legacy Data Import: NT$30,000, Lump Sum" — How the Cheapest Line on Your Quote Ate 47 Working Days

1. Line 14 of the quote

A 40-person mechanical and electrical engineering firm commissioned a client and site management system: NT$1.8M, five months. Line 14 read "Legacy data import, lump sum, NT$30,000." We only received the data in week nine: three Excel workbooks, one Access file, an MSSQL backup from a 2013 ERP. The customer master held 18,742 records; 11,206 survived deduplication, and in 4,300 the real contact sat in the notes field. That line took 47 working days, cost NT$210,000 in change orders, and pushed the project back six weeks.

One admission: we quoted NT$30,000 without looking at a single real record. That is the vendor's fault, not the client's.

2. Five myths

• "Data import is a technical problem." Eighty percent of it is not: deciding whether two "Wang Da-ming" records are the same customer is a business call only you can make.
• "Our data is clean." Being in a system is not being structured. MIT Sloan Management Review puts the cost of bad data at 15% to 25% of revenue for most companies.
• "We just import it once." The professional pattern is two to five trial loads with diff reports, then one production load, then four weeks of parallel running.
• "It's a small number, a lump sum is fine." A lump sum defers the uncertainty into a change order, which arrives once you can no longer switch vendors.
• "Obviously we migrate everything." The mainstream "complete migration" advice is wrong: migrating everything is the most expensive, least valuable option. Split the data into needed at go-live, frozen for lookup, and discarded — typically only 30% to 40% belongs in the new system.

3. The framework: Data Readiness Score and hours formula

Five dimensions, 0–4 points each, 20 maximum:

H (migration hours) = (T × 4) + (F × 0.5) + (N × R × 0.03) + (C × 8)

T = entities, F = field mappings, N = records, R = dirty-data rate, C = trial-load rounds. The DRS sets R and C: 17–20 means C=2 and R at or below 5%; 13–16 means C=3, R 5–15%; 9–12 means C=4, R 15–30%; 5–8 means C=5 and dual-write parallel running; below 4, clean up before starting.

The engineering firm scored DRS 10 with T=9, F=180, N=18,742, R=22%, C=4: 36 + 90 + 124 + 32 = 282 hours, or NT$451,200 at NT$1,600 an hour. The gap — roughly NT$240,000 — we absorbed by compressing testing and documentation.

4. Three companies, three conclusions

The same formula tells the third company not to do what it planned.

5. The hidden cost list (engineering firm)

• Client-side adjudication, 58 hours: 34 hours confirming fields, 24 ruling on duplicates. All on your staff.
• Joint mapping meetings, 68 hours: 9 entities × 2.5 rounds × 1.5 hours × both sides.
• Parallel running, 144 hours: 6 people × 1.2 hours a day × 20 working days, about NT$50,400 of payroll.
• Test environment and anonymisation, NT$15,000–35,000: under Taiwan's Personal Data Protection Act, personal data should not land in staging as-is.
• Encoding and rare characters, 8–20 hours: converting Big5 to MySQL utf8mb4 leaves rare characters and emoji no mapping table resolves.
• Excel's physical ceiling: legacy .xls sheets cap at 65,536 rows; anything beyond was silently truncated years ago.
• Extended legacy licences, NT$4,000–20,000 a month: no early cancellation during parallel running, so budget three to six months.
• Post-launch correction batches, 10.5 hours: 2.1 batches on average in the first three months.

Total: NT$280,000–420,000, nine to fourteen times that "lump sum NT$30,000" line.

6. A 12-point migration scorecard

Score each 0–2, maximum 24, and ask your shortlisted vendors:

• Insists on seeing 200+ real records before quoting
• Names the field mapping sheet as a deliverable
• Writes down deduplication rules and the decision owner
• Schedules three or more trial loads with diff reports
• Provides a reconciliation table (source = imported + discarded + pending)
• Provides a pre-import snapshot and rollback plan
• Anonymises personal data in the test environment
• Produces a "not migrating" list for you to sign
• Includes a data quality report (null, duplicate, format anomaly rates)
• Delivers idempotent, re-runnable import scripts
• Plans a parallel-running period with dual reconciliation
• Makes data acceptance its own payment milestone

19 or above, sign. 13–18, demand the missing clauses. 12 or below, change vendor — or just buy a readiness audit first.

7. What we offer, and what we turn down

ScriptWalker splits migration into four models: a data readiness audit (NT$45,000–80,000, 10 working days, delivering the DRS score and an hours range, credited against a later project); a fixed-price project (fixed for scripts, unit-priced for trial loads and adjudication); a retainer (three months of post-launch correction batches, from NT$25,000 a month); and an advisory model (we design the rules, your IT executes).

Where we are the wrong vendor:

• Clients who insist on a lump sum and will not let anyone look at the data.
• Companies with nobody authorised to rule on data disputes within 48 hours.
• Anyone demanding 100% migration with no frozen archive.
• Data containing personal information collected without consent, where compliance work is off the table.
• Cases where the incumbent refuses database access and you have no contractual basis — that needs a lawyer.

8. The 90-day migration playbook

• Days 0–10: sample 200–500 records, score the DRS, produce and sign the "not migrating" list.
• Days 11–30: freeze the mapping sheet, confirm deduplication rules in writing, run trial load 1 with one goal — that it completes.
• Days 31–60: trial loads 2 and 3, a diff report after each, a 90-minute weekly adjudication meeting to clear the pending list.
• Days 61–75: production dress rehearsal with snapshot and rollback drill, plus the reconciliation table.
• Days 76–90: cut over, run four weeks in parallel, hold a day-90 data health review (null rate, duplicate rate, user-reported issues).

9. Decision checklist

• ☐ I can name how many systems the legacy data comes from
• ☐ I know the real record count of the customer master
• ☐ I have a rough duplicate rate
• ☐ Key information is not buried in notes fields
• ☐ Someone can rule on whether two records are the same entity
• ☐ I accept that only 30–40% may be migrated
• ☐ I have decided what only needs frozen lookup
• ☐ The vendor saw real samples before quoting
• ☐ The contract names a mapping sheet and reconciliation table
• ☐ Trial-load rounds and diff reports are in the schedule
• ☐ Data acceptance has its own payment milestone
• ☐ Headcount is budgeted for four weeks of parallel running
• ☐ The legacy system can be extended three to six months

Ten or more ticked: go to migration. Fewer than seven: a readiness audit is cheaper.

Frequently asked questions

The vendor says a lump sum for data import is fine. Should I push back?

Yes. Ask for a fixed price on scripts, a unit price per trial-load round, and a unit price per 1,000 records of adjudication.

Can I clean up the spreadsheets myself first?

Yes, and it pays: standardise date and phone formats, and pull contact names out of notes into their own column. That drops R from above 20% to below 10% and cuts about a third of the hours.

If we do not migrate old data, can we still look it up?

Yes. Convert the legacy database into a read-only archive with a lookup interface, roughly NT$40,000–90,000.

Can a bad migration be reversed?

With a complete pre-import snapshot and idempotent scripts, a rollback usually finishes inside two hours.

Is money spent on a readiness audit wasted?

No. The DRS score and "not migrating" list are portable — any vendor can quote from them.

Next step

Holding a quote with a line that reads "legacy data import, lump sum"? Do not sign it yet. We offer a free 30-minute consultation: we will run the DRS and the hours formula with you on the spot, or go straight to the 10-day readiness audit.

• Email: [email protected]
• Phone: 0916-224-047
• LINE: @ufv9089p