Ive got a bit of a problem here.
WE just picked up a new IBM san and hooked it up with 10gbE via ibms recommendations and best practices.
Here is my situation;
Hosts
HP dl360 g7 servers currently connected over 1gbe to a older san (we are trying to migrate away from)
Each server runs esx 4.1
Vswitch0 = service console
Vswitch1 = 1gb vmotion (on its own vlan/subnet)
Vswitch2 = 1gb vm network (on its own vlan/subnet)
Vswitch3 = 1gb storage network (on its own vlan/subnet)
Vswitch4 = 10gbe storage network (isolated HP 10gbe switch)
Our 10gbE cards in our hosts are HP NC552sfp running the latest vmware drivers (4.1.334.0) and firmware (4.1.450.16)
I am utilizing the vmware softrware iscsi adapter. All of our data is in vmdk files. No in guest mapping to the storage.
Now my question
I cant seem to get past 1gb speeds to our new storage.
For instance, if i svmotion 2 VMs to the new storage on a new volume and also vmotion the 2vms so they run off the same host.....and
I try to copy a 25GB file from a share on 1 Vm to the Other VM in the guest OS, the best i can get is about 60 - 80MB/sec
IF i look at esxtop i show utilization on Vswitch2 and Vsiwtch4 which makes sense but it is only at 1gb speeds.
Is there a direct relationship between your VMNics and your storage nics where you are only as fast as your slowest connection?
Even though they are on the same host and datastore, does it still use the VM netowrk and the physical nics to do the transfer or is it all internal and only utilizing the 10gbe?
Both of the VMs im testing in have the vmxnet3 adapters.
Right now ive tried JUMBO frames from my 10gbe nics, vswitch4 and new san and it was the same or worse so that didnt help.
IF i copy a 22gb file to itself the same place on the same VM my storage nics show activity. Windows reports this speed to be about 100MB/sec....or what i would expect a 1gb connection to do at full tilt.
I have tried connecting a a physical server (same make, model, hardware), loaded windows 2008r2 on it and used the MS iscsi int to mount storage and was able to copy a 12gb file in a few seconds time at 1.8+ GB/Sec.....so i know faster is possible.
what am i missing?