Well done Ted,

Making an opinion in this area can only take 2 ways: from other's experience and from your experience. No theoretical model is able to take into consideration all what you've perfectly related.

* other's experience*

Your story tells in short that rewriting an existing system throws it out of control.

In https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/, Joel Spolsky (Yale, ex MS Excel team leader and founder of https://stackoverflow.com/ which has taken over the developer forum market in a couple of years) writes: it's the "single worst strategic mistake that any software company can make".

*my experience*

I've practised 3 ways:

1- * greenfield*: write a spec describing the system from an outside point of view and hire a new development team

I've written tens of functional specs that always, at some point, were to replace an existing system. Even in the 80's, even if we designed all the forms and reports very precisely and reviewed with the users over and over until we got a validation, we were always missing critical points and migrating to the new system could not be a one-step process, but a progressive, continuous sliding through iterations. At that time companies could still run without IT; uncomfortably but possibly. Today it'd be a disaster.

2- * migration*: keep the overall structure, translate the code into another language

one can consider that the existing forms and reports are the best possible way users want to interact with the data, just because it's the result of years of work and refinement. User events being almost universal across languages, even the code structure can be kept as is, provided that the new language offers the same kind of interactions between modules (eg. pass parameters by value or reference; not that obvious: JavaScript can't pass by reference, one needs to use closure instead).

The issue is a very simple principle that Joel Spolsky writes in a few words: "It’s harder to read code than to write it."

Specifically, even if you understand what the original code intends to do, in most cases you miss a test case to check that the new code does exactly the same. Not because the previous programmer has ignored TDD, because TDD is very difficult to apply to applications driven by user interaction.

Very often you write a piece of code in response to a user demand, based on a repro scenario that was valid when the code was written, but you see no reason why you would keep trace of this repro case in the code itself. In this case, having the previous developer involved in the rewrite project can be useful provided that he has kept enough history and archives to help new developers understand the subtilities.

One possible, and I think fruitful way, is to refactor the old code prior to rewrite. May seem awkward to further invest in a code that'll finally be dumped but I've experienced that safer and faster than just giving the code to the new team and let them raise the questions; at some point they'll be tempted to say "it's useless" and just overlook some code that will later reveal important.

Another benefit from building the new system with the same structure as the old one is an improved ability to debug an issue by team working between the previous team and the new team; at least both team focus on the same modules, the same pieces of functionalities. Of course the new team dislikes being tied into an old, sub-optimal, "not invented here" structure.

3- *adaptation*:

(this is the solution FoxInCloud is based upon)

Thinking out of the box, and considering that the existing system does the job except that it does not fit into a new set of technologies, just change the pieces of the system into new pieces built with the new technologies.

The classic industry calls this 'retrofiting' or 'revamping'; eg. my first job was about installing electronic control systems (regulation and automation) on existing steel rolling mills; much cheaper than changing the whole plant, for the exact same performances.

This same idea upraised back in 1999 when a client asked us to clone an existing desktop application on the Web. Even if the application was pretty young (a couple of years) I could not see how solutions (1) or (2) could succeed; too risky and, based on gut feeling, almost impossible to ensure that both applications would provide the same results for the same user scenarios; and this could have raised commercial issues that we could have been responsible for; eg. large orders at a wrong unit price.

Thousand of VFP developers have done the same by changing a VFP database into a SQL database.

VFP Advanced moves into the same direction: change 32-bit into 64-bit, while keeping the whole rest unchanged.

I'm convinced that, at the end, technologies will adapt to systems, not the other way round.

Thierry Nivelet

FoxInCloud Give your VFP app a second life in the cloud http://foxincloud.com/

Le 24/10/2017 à 23:38, Ted Roche a écrit :

Three weeks ago, now, On Thu, Oct 5, 2017 at 3:52 PM, Thierry Nivelet tnivelet@foxincloud.com wrote:

...

Well, I often read from experts that rewriting a s/w using new techs was a piece of cake

Thierry Nivelet http://foxincloud.com/ Give your VFP app a second life in the cloud

Thierry:

I wanted to follow up on this.

Given a full set of specifications on what the old software does, And a full set of specifications of all the changes the client wants for the new software, And a full set of tests to ensure 100% compliance, it's a piece of cake. Very expensive and slow, but cake.

Fresh coding in a new language with modern tools is faster, but that's the smallest part of application development. I was amazed what kind of power I could find in PHP or Python or Ruby.

Next year, I celebrate (?) the 40th anniversary of shipping my first commercial application. I have rewritten several applications from FPD to VFP, and from VFP to PHP or Rails, and there is always undocumented behavior that turns out to be key to the application, and very complex and time-consuming to backfill into the new application when it is discovered late in the process. It's also essential that at least some of the developers have expertise both in the old language and the new one. In one case I had to explain to a client that the textbook-perfect description of the algorithm they provided me with was all very nice, but it wasn't actually how their FoxBase system had been running for the past 15 years. It's hard to tie out new results with old results when they do different math. And persnickety things like precision and rounding rules as you switch front-end languages and back-end storage make that tougher than some imagine.

But, as someone recently pointed out in another thread, it is also hard to do "blank slate" development where the client *thinks* they know what they want, but invariably, it is something else they need. In the best of all worlds, you need a full specification on what the software does, and a set of tests to show 100% compliance, and it is almost impossible to produce, until you've written the system. Clients may understand how their business works, but managers view their world through the 90% case and skim over the exceptions, while front-line workers deal with the exceptions all day and miss the big picture.

In sum, software development is *hard*.

In reality, a slow, deliberate, iterative system of incrementally building a new system from the old is the only thing I've ever seen succeed. It's hard, it's more expensive and it takes more time that anyone wants it to, but the end result can be an application the client can run with, expand their business, and compete successfully for the next 20 years.

[excessive quoting removed by server]