User Data Migration Plan

Winuser said:

I find cut and paste to dangerous.

Click to expand...

I am not sure, but because of the way the icons behave in W11 during the cut and paste operation, I'd like to think that W11 does not cut when the paste in the cut and paste operation is not successful.

Winuser said:

I do hope that you find a method that fills all of your needs.

Click to expand...

Thanks, but I am after insight in the various methods rather than anything else.

Try3 said:

John,

In Haydon's command switches, /S makes RoboCopy include non-empty sub-folders and /E makes it include sub-folders even if they are empty.
Using both /S and /E is not necessary. /E will do.

Merry Christmas,
Denis

Click to expand...

uh, see below - His script does not have /S or /E. And yet the log file does.

Haydon said:

Folks, referring to my post #13, note that in my script I had neither /S nor /E
However, the header of the log file had both /S and /E

Click to expand...

I noticed this, and to make sure, I copied both the script you had used and the logfile results. It was rather odd to see that /S and /E were there when you specifically did not specify them in the command in the first place.

I wonder if one of the *other* arguments used in the command is forcing /S and /E?

Haydon said:

Folks, again referring to my post #13, note that in both script and log file I had /XA:HST which excludes Hidden files, System files, Temporary files.

Click to expand...

Cool.

Haydon said:

Did you mean to say that after installing your apps on a clean install of W11, you do not have to configure your apps just by copying \AppData from your old W10?

Click to expand...

For a lot of apps, not *all* apps. For example:

\Users\{userid}\AppData\Local\Google\Chrome\User Data\ contains all of the Chrome profiles you have on your system.

On a new install, I install Chrome for all users, and then let it open.

1. I then close Chrome and make sure it is not running in the background (open Task Manager and end any instance of Chrome.
   
   
2. I then go to the same location \Users\{userid}\AppData\Local\Google\Chrome\User Data\ on the newly installed system and delete everything there.
   
   
3. I then copy the old \Users\{userid}\AppData\Local\Google\Chrome\User Data\ to the new system.
   
   
4. Presto - my Chrome now opens exactly like it did on the old system, no need to wait on synchronizing, for some tabs not to make it, etc. - it is *exactly* as you left it on the old system

This works with most browsers, you just have to know where each browser stores its profile information and such. Firefox is a bit special in that it usually has folders in both AppData\Local\ *and* \AppData\Roaming\. But, knowing this, it's easy to get browsers back up and running.

For other apps, it depends solely on the app. Having done this for many years now, I know which files and folders can and cannot be copied to a new system to make it work.

Take Microsoft Office, for example. The location for the settings and such is also split between AppData\Local\ *and* \AppData\Roaming\. However, some of thsoe subfolders can be copied over to retain settings, and some simply cannot. Also, most are in the subdirectory \Office\, but not all - Outlook typically has its own folder in both, Teams, as well, etc. Knowing what can be copied and restored is a big part of making efficient use of the full \AppData copy. But I do not use scripts, as I make a full copy of all \AppData fodlers (with Windows complaining here that some files are not accessible) to a different drive, and then delete \Temp folders and the like. I then pick and choose what I copy to the new system, depending upon the apps I install - but that copy of \AppData is always there until I finish installing everything that I need.

When it comes time for a new clean install, I can repeat the process, but the older one is also still there in case I did not install an app from a previous install, but in the next clean install I do plan to do so.

This could be accomplished using a number of techniques - CopyPasta (lol), Backup program, scripting through RoboCopy / XCopy, TeraCopy and other file copier apps, and other ways. I stick to the copy and paste as I find it more efficient to copy just what I need at any given moment rather than copying everything in one fell swoop.

Also, come apps don't like it when you pre-copystuff, and will simply delete it and start over anyway. So, with some apps installing the app first is required, then you remove the existing info and copy the new (old) info there. Firefox was notorious for this, and it's one of the reasons that I resorted to installing apps then copying settings data after installation.

In this respect, restoring data from image backups performs least well as when I want a truly clean install, an image restore is not going to be any cleaner than when it was made. And using Imaging software to do this becomes cumbersome, partially because of my own idiosyncrasies in having used my own method for so long now. But it can surely be done, so my take on it is purely subjective. And, for similar reasons, scripting is never going to satisfy me either as a copy and paste of the entire tree under AppData works perfectly fine with maybe about a 100 prompts that files are not available. But I don't see an easy script that will allow me to copy singular trees and only copy some subfolders in some cases, all subfolders in other cases. But the time I got something like that set up, with all of the prompts to configure it for every app, it just makes more sense to manually restore.

Haydon said:

I am not sure, but because of the way the icons behave in W11 during the cut and paste operation, I'd like to think that W11 does not cut when the paste in the cut and paste operation is not successful.

Click to expand...

Icons of what, exactly?

The icons of files and folders involved in the cut and paste process.
It looks like a cut after paste process.

@Try3 can read my posts, BTW, you don't have to repeat my posts.

johnlgalt said:

It was rather odd to see that /S and /E were there when you specifically did not specify them in the command in the first place.

I wonder if one of the *other* arguments used in the command is forcing /S and /E?

Click to expand...

Yes, I think that /MIR puts them there.
{Haydon pointed out that I had mistakenly read the switches line in the log header instead of in his command line itself.}

Haydon said:

@Try3 can read my posts, BTW, you don't have to repeat my posts.

Click to expand...

I really don't understand that sentence.
- Why 'mention' me?
- Of course I can read your posts.
- I quote extracts that help to set the context for my responses.


My reasons for preferring RoboCopy to other methods, such as copy n paste, are rather buried in history.
I've preferred RoboCopy for so long now that it is difficult to remember why. It's become habitual. I know that amongst the reasons are:
- RoboCopy preserves the a bit and I use that in some backup scripts.
- RoboCopy preserves file creation dates. They can help me understand apparently contradictory information in files.
- RoboCopy, when used with appropriate switches, avoids an awkward mixture of permissions on files synced to my other computers.


I too backup/transfer selected bits of AppData.
- My phone syncing utility, MPE, saves my settings within AppData so I can set up a new phone and/or modified settings for an existing phone on just one computer then transfer them to my other computers or other user accounts.
- Similarly, as John said above, various Outlook configuration files are saved within AppData. Backing them up with RoboCopy is worthwhile.
- MSOffice Quick Access Toolbars, the mini-toolbars that can be put up in the TitleBar row, are also saved within AppData.
- Current-user Start menu %userprofile%\AppData\Roaming\Microsoft\Windows\Start Menu\Programs
- Taskbar pinned items %userprofile%\AppData\Roaming\Microsoft\Internet Explorer\Quick Launch\User Pinned\TaskBar
- SendTo entries %userprofile%\AppData\Roaming\Microsoft\Windows\SendTo



All the best,
Denis

@Try3 Please disregard the miscommunication, thanks!

Back to this thread, it produced some discussion on copy and paste (item #1 in the OP) and a lot of discussion on RoboCopy (item #3 in the OP) but there was no discussion on item #2 in the OP, File and Folder backup/restore using Macrium Reflect Home (MRHFF).

I have done only limited experimentation with MRHFF and I wonder if there are others here who have done more in this regard?

I can confirm that MRHFF does work, like:
> backing up your entire user folder in a .mrbak file presumably with all metadata intact
> excluding from the .mrbak file Hidden files, System Files, Temporary files, other file types
> restoring from the ,mrbak file your entire user folder OR restoring individual files and individual subfolders one at a time, presumably with all metadata intact

What I don't know is how well #2 (MRHFF) fares in comparison to #1 (copy and paste) and #3 (RoboCopy). I'd like to stimulate the discussion on the issue

For example, MRHFF probably migrates all the crud from source to destination, but I am not too concerned about a couple of GBs of crud, because I created 80 GB worth of crud by mistake and it did not affect performance at all.

etc., etc.

I only use MR for full system images so cannot comment on its use for specific folders.
I use RoboCopy for that.

Best of luck,
Denis

Try3 said:

Yes, I think that /MIR puts them there.

Click to expand...

Ahhh, That makes sense.

Try3 said:

My reasons for preferring RoboCopy to other methods, such as copy n paste, are rather buried in history.
I've preferred RoboCopy for so long now that it is difficult to remember why. It's become habitual. I know that amongst the reasons are:
- RoboCopy preserves the a bit and I use that in some backup scripts.
- RoboCopy preserves file creation dates. They can help me understand apparently contradictory information in files.
- RoboCopy, when used with appropriate switches, avoids an awkward mixture of permissions on files synced to my other computers.


I too backup/transfer selected bits of AppData.
- My phone syncing utility, MPE, saves my settings within AppData so I can set up a new phone and/or modified settings for an existing phone on just one computer then transfer them to my other computers or other user accounts.
- Similarly, as John said above, various Outlook configuration files are saved within AppData. Backing them up with RoboCopy is worthwhile.
- MSOffice Quick Access Toolbars, the mini-toolbars that can be put up in the TitleBar row, are also saved within AppData.
- Current-user Start menu %userprofile%\AppData\Roaming\Microsoft\Windows\Start Menu\Programs
- Taskbar pinned items %userprofile%\AppData\Roaming\Microsoft\Internet Explorer\Quick Launch\User Pinned\TaskBar
- SendTo entries %userprofile%\AppData\Roaming\Microsoft\Windows\SendTo

Click to expand...

All good reasons. I suppose I never bothered with developing RoboCopy because when I kept trying to use it, I invariably ran into issues (and these were genuine PEBKAC issues, not RoboCopy itself) that I invested in additional storage to use for direct copypasta. Certainly there is nothing wrong with any of the methods Hayden mentions. To each their own - whatever works for everyone, more power to you for using it.

Haydon said:

What I don't know is how well #2 (MRHFF) fares in comparison to #1 (copy and paste) and #3 (RoboCopy). I'd like to stimulate the discussion on the issue

Click to expand...

As stated before, I definitely don't use traditional backup programs to backup and restore individual files. I use them for emergency recovery of existing systems. Just like Denis prefers to use RoboCopy for individual file / folder restoration (particularly \AppData stuff) I prefer to not use a backup program, preferring plain old copy and paste.

But,. hopefully other users can chime in on their methodology specifically using MR or other imaging software to restore \AppData and the like onto new installs.

Here is a bit of further stimuli for discussion, a few more thoughts on MRHFF (Macrium Reflect Home for File and Folder backup/restore)

You need MRHFF on both source and destination machines, but not at the same time, so you could theoretically activate/deactivate a single (paid for) license. I found that impractical (it also felt a bit like cheating) so I used a free one month trial for my experiments. I'd like to think though that for doing a real migration you do need 2 licenses so you have stable conditions at both source and destination in case of trouble. (RoboCopy and 'copy and paste' are built into the OS and are always available at both source and destination)

From the limited amount of experimentation that I did, I seem to notice that a restore from a full .mrbak file works best, or to say it the other way around 'full + differential/incremental' is worse for doing a migration.

What I found nice is the file and folder count, and file and folder size info, for both the backup phase and the restore phase (so you can match the latter to the former to verify the migration) just like what RoboCopy allows you to do with its log files, but MRHFF allows you to do it with a nice GUI. What I found not so nice is that MRHFF and RoboCopy produce somewhat different numbers! It raises the question, what exactly do these 2 venerable tools actually migrate? Are the different numbers merely a result of different ways of counting, or is it something more substantial? I sure hope it is not the latter, but how to verify? On the other hand, there seems to be no mention of people losing file and folders in a migration (at least not regularly) and both tools have been around for decades.

I reread the last paragraph of my previous posting, and perhaps it is a bit too strongly worded. MRHFF and RoboCopy are 2 different apps and as such they can simply not be configured identically.

For example, in MRHFF, the migration options that I used for experimentation are:
Exclude System files
Exclude Hidden files
Exclude file filter: *.temp; *.tmp; *.bak; *.~*
There is no explicit exclusion of junction points, but MRHFF must have implemented it, in some way.

In RoboCopy, the exclusions in my script are as per the definitions in the MS reference at the bottom of this post
/XA: Excludes files for which any of the specified attributes are set.
S - System
H - Hidden
T - Temporary
/XJ Excludes junction points (which RoboCopy normally includes by default!)

Are the above differences enough to explain the different numbers?
I don't know

Do MRHFF and RoboCopy migrate all user data of any significance?
Most likely, but on the margins it depends on the definition of significance, I guess.

Maybe I will join the 'copy and paste' movement too, in blissful ignorance

You'll generally find it better to refer to the explanations in
www.ss64.com
rather than those in MSLearn [previously MSDocs, previously TecnNet].

SS64 generally provides better explanations.


Denis

I find it very interesting that johnlgalt and Try3 completely ignored Winuser, Porthos, and myself on this post was just between them. And every other interjection from another member of this forum didn't matter or had no relevance in this post. Sorry but I find this offensive. Maybe I'm being sensitive, but I'm not thinking so. I think this is purposeful and controlling. Thank you to johnlgalt and Try3 for being long-winded and not adding anything another member has to say to this post no matter how relevant or irrelevant they are. We may not be as smart as you "Spkeeking for me" But we all contribute. Maybe not so long-winded but to the point as it should be. So go ahead... Beat me up on this one.

You, Winuser, Porthos said what you wanted to say.
I said what I wanted to say.
John said what John wanted to say.
Haydon said what Haydon wanted to say.
Where's the beef?

Please note that I did not accuse anybody of anything and I did not insult anybody.

Denis

Try3 said:

You, Winuser, Porthos said what you wanted to say.
I said what I wanted to say.
John said what John wanted to say.
Haydon said what Haydon wanted to say.
Where's the beef?

Please note that I did not accuse anybody of anything and I did not insult anybody.

Denis

Click to expand...

The fact that you are on the higher end of this forum and Ten. You've been around so so long. You should give recognition to those that try and contribute. Contribution is nothing without recognition.

What exactly is it that you want? Be specific.

Denis

Try3 said:

You, Winuser, Porthos said what you wanted to say.
I said what I wanted to say.
John said what John wanted to say.
Haydon said what Haydon wanted to say.
Where's the beef?

Please note that I did not accuse anybody of anything and I did not insult anybody.

Denis

Click to expand...

Hi, Denis:

No, you didn't accuse anyone of anything, @Try3.

It's my understanding that there is no requirement in thread protocol for those helping the OP (in this case @Haydon) to respond to others also trying to help. In fact, if multiple entities are helping, it would be rude to disagree with what someone else was suggesting unless it was so totally wrong that the computer in question might be messed up. Of course, that hardly ever happens, but I have seen a guru or two have to disagree. Best to leave it lie if possible, though . . .

In fact, @Winuser is advocating copy and paste. The OP @Haydon responded, giving his reasoning whether or not copy and paste would work, or not, for him. This is the response called for . . . Since Winuser was addressing @Haydon . . . Since every response I saw was pointed at Haydon, and since he's the OP, protocol calls for him to decide whether to address the suggestion.

@johngalt jumped in to help early on, and you followed suit shortly thereafter. It got too complicated for me to follow, but you two are doing what you can to help Haydon understand what he wants to do. Haydon is following up with new info and such, which you and John are helping with.

Wynona said:

It's my understanding that there is no requirement in thread protocol for those helping the OP (in this case @Haydon) to respond to others also trying to help. In fact, if multiple entities are helping, it would be rude to disagree with what someone else was suggesting unless it was so totally wrong that the computer in question might be messed up.

Click to expand...

This.

Haydon said:

You need MRHFF on both source and destination machines, but not at the same time, so you could theoretically activate/deactivate a single (paid for) license. I found that impractical (it also felt a bit like cheating) so I used a free one month trial for my experiments. I'd like to think though that for doing a real migration you do need 2 licenses so you have stable conditions at both source and destination in case of trouble.

Click to expand...

Way back in the day when I tried manual restoring, this might have come up as a limitation for me as well. It's not so bad if you have a family pack with unused licenses, but for a single license, moving to a new device, this kinda stuck in my craw a bit.

Not so bad when you are cleanly installing on the same device, but moving from an old device that you plan to wipe to a new device and using a single license, while following the licensing terms, can be a bit rough.

Haydon said:

MRHFF and RoboCopy are 2 different apps and as such they can simply not be configured identically.

Click to expand...

That was my initial thought in reading the previous post as well. You covered it in this post pretty well, though.

Haydon said:

Do MRHFF and RoboCopy migrate all user data of any significance?

Click to expand...

With MR, if you do a complete backup then restore it back, you should be able to tell if it does or not. Of course, you'll want a non-mission-critical system to test on, but generally, the vast majority of Macrium users here are pretty happy with it when restoring, particularly when something goes goofy, so I'd venture that it does copy nearly everything.

As for RoboCopy - well, it will copy what you tell it to. Figuring that part out is the headache lol.

Haydon said:

Maybe I will join the 'copy and paste' movement too, in blissful ignorance

Click to expand...

Ha! Next thing you'll be telling us you're going back to DOS permanently!

Try3 said:

You'll generally find it better to refer to the explanations in
www.ss64.com
rather than those in MSLearn [previously MSDocs, previously TecnNet].

SS64 generally provides better explanations.


Denis

Click to expand...

100% agreed with this. That site has been a godsend for me personally sine I first discovered it a long, long time ago.