what's the best way to read write to from a compact flash card?

[Miguel2013]

I was copying data to the CF card using a usb 3.0 reader but notice some slow downs when copying lots of tiny files then I thought it's cause is using USB.
Is there a -esata CF reader or -pcie card reader or a better method then usb even if it wasn't usb fault. What's the best method for using compact flash.

 

[nomdeplume]

USB is by far the best. Most USB readers are far from the best.

Sounds like overheating or poor implementation. Good readers are not cheap since they target the pro photo and other commercial environments. For slower CF cards you have a bit more leeway.

Larger truth is lots of small files are always slow no matter what storage.

 

[Solaris17]

Larger truth is lots of small files are always slow no matter what storage.

This is the answer many small transfers kill all throughput.

 

[Miguel2013]

would u recomend me an expensive cf reader/writer?

 

[nomdeplume]

would u recomend me an expensive cf reader/writer?

What is your primary use here? What sort of speeds, how often and how intensive will you be using it?

Also cost should never be equated with quality.

 

[Miguel2013]

What is your primary use here? What sort of speeds, how often and how intensive will you be using it?

Also cost should never be equated with quality.

I just got 1 large CF card 128GB and plan getting a 512GB for back ups of folders, I tried writing a pictures folder where most files are between 500kb to 10mb/s to and from the same drives nvme and sata3 ssd and both disks did good with avg of 500MB/s while for the usb reader with the CF card from the ssd to cf it did avg 30-40MB/s. the cf card isn't too bad it has consistent 87MB/s on sequential writes and more for reads.

as I was writing this it figure I need a better CF card brand and model and this one is generic. Then I'll get back if I find the CF still can't read faster.
by the way I have this reader: https://www.ebay.com/itm/194298904906

 

[nomdeplume]

I just got 1 large CF card 128GB and plan getting a 512GB for back ups of folders, I tried writing a pictures folder where most files are between 500kb to 10mb/s to and from the same drives nvme and sata3 ssd and both disks did good with avg of 500MB/s while for the usb reader with the CF card from the ssd to cf it did avg 30-40MB/s. the cf card isn't too bad it has consistent 87MB/s on sequential writes and more for reads.

There can be a rather large divide between a lower performing off brand and one of the better offerings from a name brand. Non exclusive, you could have a perfect data integrity and speed now being held back by an all-in-one dongle.

as I was writing this it figure I need a better CF card brand and model and this one is generic. Then I'll get back if I find the CF still can't read faster.
by the way I have this reader: https://www.ebay.com/itm/194298904906

I'd certainly suggest a dedicated CF reader, but nothing too overboard. That one doesn't inspire confidence.

Don't look past a simple circuit reader barely larger than the card for your needs if the construction and electronics seem to have potential. Can be a bit of a pain if there is no way to get inside to reset pins that pushed out or forcibly cool it off. Take seriously my warning about money not buying much for some ways upwards unless you need warranty and irrevocable guarantees of data integrity. I mean just don't rub it on polyester clothing and carpeting until your hair stands up and then insert the card.

 

[Logic]

1st:
Always go for AUSP enabled card readers and drive enclosures.
UASP stands for USB Attached SCSI Protocol.
This enables parallel processing of data (way faster) and thus Native Command Queuing (NCQ) and also enables TRIM (the SCSI equivalent) just for a start!

Buyer beware:
A good # of UASP enabled drive enclosures are indeed UASP capable, but are USB to SATA2 (300MB/s) not SATA3 (600MB/s)
Thats fine for a HDD where you only lose a bit of burst speed from/to the drives DRAM buffer, but is a serious bottleneck to today's SATA SSDs!
Similar 'tricks' are probably in play to sell cheaper USB to SD Card ctlr chips and USB to NVMe ctlr chips.
ie: Make sure the ctlr chip translates the fastest USB protocol your MB can do into Card Reader or NVMe protocols of a similar speed, so that neither protocol is a bottleneck...


2nd:
If your reader (or enclosure) is NOT AUSP capable; all is not lost:

USB sends data in packets, with control data attached to each packet.
The std packet size is 64KB of data + control data.
If you increase the packet size; you get more data per packet, decreasing overhead.
This can give you up to ~30% faster transferes for large files.

See MaximumTransferLength since Windows 7 SP1 here:
Drive Tools for Windows (uwe-sieber.de)
for the why's and wherefore's and an easy way to set your registry to do this.
(I am particularly proud of the USBSTOR MTLT tool as it was I who brought this tweak to Uwe Sieber's attention. He then wrote the app)

Packet size choices are 64KB, 128KB, 256KB, 512KB, 1MB, 2MB.
For best performance you want to match this size to the average file size being transferred.

ie:
For running an OS off of USB;
stay with the default 64KB as OS I/O is ~68% random I/O, NOT the #s waved around like burning flags by drive marketing..!!!

For movies etc; go with the max packet size of 2MB.

A good compromise is 512KB as used by the UASP protocol and chosen to match the Erase Block Size of most NAND chips at the time.
Matching to the NAND's EBS can in fact be faster overall than 64KB for random I/O. It all depends on the ctlr, so testing is reqd if you want the absolute best speed.


3rd:
Windows only pretends to enable write caching on USB attached storage unless the drive is formatted NTFS..!
Again: Windows Write Caching to USB is a lie, Unless the drive is formatted NTFS!!!

This limitation can be overcome with another of Uwe Sieber's tools: USB-WriteCache V0.2
This tool is available at the same link as above:
Drive Tools for Windows (uwe-sieber.de)

Now this tool will speed up large transferes a bit, but really shines when you are trying to run apps or an OS off of a flash drive where there's a lot of random writes going on.
It will often allow a drive that fails the ReadyBoost compatibility test to pass it.
That means that the R4K read/writes, which is what ReadyBoost caches, OSs do 68% of the time, apps do, etc is greatly improved overall vs the unapplicable results you'll see in popular benchmarking software.