Which device is "faster "?
-
@JonB said in Which device is "faster "?:
I have to sit there watching its light flash...
There are many reason why it is slow:
- the FLASH is "old", and there are many bad blocks and the wear leverage algorithm has many work to do (USB-strick internal firmware)
- the used file system is not adapted: many USB dongle are design to be used with FAT/FAT32 file system
- on host, the file system caches are not "optimal"
-
@KroMignon
I have just typed in some timings above. In a word, if the stick write is twice as long as the "disk" (I actually have SSD) write is that actually not a bad speed at all?@JonB said in Which device is "faster "?:
is that actually not a bad speed at all?
You are trying to compare two different technologies: you SSD which use (m)SATA (up to 600MB/sec), PCI-e (up to 32GB/sec) or M.2 (up to 4GB/sec) interface and USB which I suppose is an USB2.0 full speed one (480Mbit/s ==> 60MB/sec)
And the used FLASH technology, to do the storage, is at least not the same.
You are using Linux, but what is the used file system for your USB storage?
How did you mount the USB storage: do you use "sync" in mount option, which is a performance killer for USB devices?It is an old maxim of mine that when you have excluded the impossible, whatever remains, however improbable, must be the truth. (Sherlock Holmes)
-
@JonB said in Which device is "faster "?:
is that actually not a bad speed at all?
You are trying to compare two different technologies: you SSD which use (m)SATA (up to 600MB/sec), PCI-e (up to 32GB/sec) or M.2 (up to 4GB/sec) interface and USB which I suppose is an USB2.0 full speed one (480Mbit/s ==> 60MB/sec)
And the used FLASH technology, to do the storage, is at least not the same.
You are using Linux, but what is the used file system for your USB storage?
How did you mount the USB storage: do you use "sync" in mount option, which is a performance killer for USB devices?M.2 is a four lane PCI-E, which is the whole point of having it in the first place; you want the SSD to be on the bus.
-
@JonB said in Which device is "faster "?:
is that actually not a bad speed at all?
You are trying to compare two different technologies: you SSD which use (m)SATA (up to 600MB/sec), PCI-e (up to 32GB/sec) or M.2 (up to 4GB/sec) interface and USB which I suppose is an USB2.0 full speed one (480Mbit/s ==> 60MB/sec)
And the used FLASH technology, to do the storage, is at least not the same.
You are using Linux, but what is the used file system for your USB storage?
How did you mount the USB storage: do you use "sync" in mount option, which is a performance killer for USB devices?@KroMignon
You (I, actually!) have opened a whole can of worms here :) I realize I have hijacked this thread, and should probably take it to Lounge if I want to pursue.Since you've asked, I'll just answer briefly. It's a Linux VM guest (VirtualBox) 64-bit on 64-bit Windows 7 host. Filing system is
fusefrom Linux point of view, it's NTFS formatted from Windows so that I can access stick from Windows too if I want to. (From what you say about FAT32, maybe this is not optimal?) I don't think the Linux mounts it sync, because as I say the red light continues to flash after thattarhas completed if I try to shut down the Linux or if I wait a while after writing to it, as I would expect cached-non-sync to do.mount:/dev/sdb1 on /media/jon/HOME_STICK type fuseblk (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other,blksize=4096,uhelper=udisks2)I realize that goodness knows what's going on with VB drivers through Linux to Windows, the non-native Linux filing system etc.. What I should do is start by doing all my timings back in Windows before comparing against Linux.....
-
@KroMignon
You (I, actually!) have opened a whole can of worms here :) I realize I have hijacked this thread, and should probably take it to Lounge if I want to pursue.Since you've asked, I'll just answer briefly. It's a Linux VM guest (VirtualBox) 64-bit on 64-bit Windows 7 host. Filing system is
fusefrom Linux point of view, it's NTFS formatted from Windows so that I can access stick from Windows too if I want to. (From what you say about FAT32, maybe this is not optimal?) I don't think the Linux mounts it sync, because as I say the red light continues to flash after thattarhas completed if I try to shut down the Linux or if I wait a while after writing to it, as I would expect cached-non-sync to do.mount:/dev/sdb1 on /media/jon/HOME_STICK type fuseblk (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other,blksize=4096,uhelper=udisks2)I realize that goodness knows what's going on with VB drivers through Linux to Windows, the non-native Linux filing system etc.. What I should do is start by doing all my timings back in Windows before comparing against Linux.....
Wish you good luck to optimize your backup process!
There are many software layers which are involved ;)It is an old maxim of mine that when you have excluded the impossible, whatever remains, however improbable, must be the truth. (Sherlock Holmes)
-
Wish you good luck to optimize your backup process!
There are many software layers which are involved ;)@KroMignon
You are absolutely right. But speed is probably only a minor problem. About half the time the Linux guest does not get the USB at all from the Windows host, though the host has passed it on and cannot access it, but can't get it from Linux either. And the only way is to reboot host (not just guest) and try again till it does. Yet no such problem to stick from Windows when no Linux guest fired up. And yes I've asked at VirtualBox and they don't know what to diagnose is going on. So I have plenty of headaches :) Thanks for your comments. -
Wow
what an interesting response to silly, out of boredom , post.
Not a single "RTFM"!
Maybe this is the best sub-forum to post technical stuff disguised as "beer talk"... -
@KroMignon
You (I, actually!) have opened a whole can of worms here :) I realize I have hijacked this thread, and should probably take it to Lounge if I want to pursue.Since you've asked, I'll just answer briefly. It's a Linux VM guest (VirtualBox) 64-bit on 64-bit Windows 7 host. Filing system is
fusefrom Linux point of view, it's NTFS formatted from Windows so that I can access stick from Windows too if I want to. (From what you say about FAT32, maybe this is not optimal?) I don't think the Linux mounts it sync, because as I say the red light continues to flash after thattarhas completed if I try to shut down the Linux or if I wait a while after writing to it, as I would expect cached-non-sync to do.mount:/dev/sdb1 on /media/jon/HOME_STICK type fuseblk (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other,blksize=4096,uhelper=udisks2)I realize that goodness knows what's going on with VB drivers through Linux to Windows, the non-native Linux filing system etc.. What I should do is start by doing all my timings back in Windows before comparing against Linux.....
@JonB Funny it wasn't mentioned but a single huge throttle to your throughput I will sum up in one word FUSE - filesystem in user space. All that IO is being managed in userspace but still requires copies to/from kernel land to do the low level disk IO. As for flash technologies and their features/limitations...I'm not even gonna get started. LOL
The dystopian literature that served as a warning in my youth has become an instruction manual in my elder years.
-
@JonB Funny it wasn't mentioned but a single huge throttle to your throughput I will sum up in one word FUSE - filesystem in user space. All that IO is being managed in userspace but still requires copies to/from kernel land to do the low level disk IO. As for flash technologies and their features/limitations...I'm not even gonna get started. LOL
@Kent-Dorfman
Interesting, but quite beyond me :) "Hardware"/"driver" issues like this is not my area!Here's my take, as a layman. All I am copying is a large (0.5GB) single file --- not lots of small ones, so we' may be talking about block allocation but not many directory entries allocation. There is a physical stick in the port. It is "slow". I don't really know whether that is slow at the USB port, or (more likely) slow in the physical stick. "Filesystem in user space", copying to/from the kernel, anything else to do with memory/CPU access == fast, comparatively. You could probably put The Treacle File System on that stick and it would make little difference. The only real slowness comes when something tries to physically push a byte out onto that stick.
Is that not right? I am interested in these things, but know nothing :)
-
@Kent-Dorfman
Interesting, but quite beyond me :) "Hardware"/"driver" issues like this is not my area!Here's my take, as a layman. All I am copying is a large (0.5GB) single file --- not lots of small ones, so we' may be talking about block allocation but not many directory entries allocation. There is a physical stick in the port. It is "slow". I don't really know whether that is slow at the USB port, or (more likely) slow in the physical stick. "Filesystem in user space", copying to/from the kernel, anything else to do with memory/CPU access == fast, comparatively. You could probably put The Treacle File System on that stick and it would make little difference. The only real slowness comes when something tries to physically push a byte out onto that stick.
Is that not right? I am interested in these things, but know nothing :)
@JonB said in Which device is "faster "?:
The only real slowness comes when something tries to physically push a byte out onto that stick.
Is that not right? I am interested in these things, but know nothing :)To fulfill this, my (last) 2 cts.
I am also had speed limitations on file transfer between VirtalMachine Guest (Linux/Ubuntu) <=> Host (Windows 7/10).
One trick what helps me was to use SAMBA, which transfer rates was much more better as Shared Directory (with VirtualBox).But this implies 2 transfers, on from Guest to Host and then vom Host to USB-Stick, it is not so smart but quicker ;)
-
This post is deleted!
