Adventures in Amazon and eBay

So as a back-end developer here at Football Fanatics, I was originally brought on to create or redo the "feed" process which ties into the Amazon API. Since then we have added the eBay API interface. I have done all this with a framework already established using WCF/WF and the Enterprise Application Blocks from MS.

Ouch, these have been tough. The framework has remained stable and posts several million requests either to the database or via SOAP everyday, it's the Amazon and eBay API and subsequently the support from Amazon and the third party eBay API we are using. We don't post directly to eBay we use another company called Channel Advisor which post to eBay for us.

First learning lesson, despite all my complaining about Amazon, from a programmatic standpoint the process that Amazon has developed is at least able to handle a load. I say this with a frown because Amazon uses java. Originally being a java developer, I both understand it and appreciate it, although I think Amazon sometimes goes overboard with the granularity of the process at least the system works. Now for the bad part. The Channel Advisor (CA) system for eBay is written in .NET. They are using what appears to be .NET 2.0 which is good, but the real problem comes in with volume. I am constantly returned errors from their database about victim of deadlock. Obviously being a SQL error, I am a firm believer that a customer should never see this. This says some pretty bad things about the design of your system.

Now I say this to say the following, the .NET system process more on demand and more rapidly in most cases. Why is this?

I have some speculation knowing both java and .NET but I am not writing to discuss memory utilization or anything else. In my effort to upgrade to 4.0 and I am going to show some of the solutions I use to solve problem with the way both the Amazon system and the eBay system interact with SOAP requests and individual things that eBay and Amazon don't necessarily document well. Hopefully this will help just one person.

Amazon has an interesting system where by you post the individual micro pieces that make up a "product". For instance you post images in an image feed, prices in a prices feed and quantity in a quantity feed and such, where with eBay you post everything together. So with Amazon you have to post and then go back and check the status of that post. This presents a problem in that you have to match up what you posted with what you store locally and then handle the error issues separately. However, with the CA eBay feed you send the response immediately and get a response immediately. This poses a problem because you can only send a small quantity of updates at a time and it takes a very long time to respond. This has caused issue with WCF timeouts and means that we have to manage long running processes in order to handle the additional time it takes to run a smaller quantity of products through.

One of the first issues I dealt with was timeouts. This is a tough subject in WCF, kinda. It's not tough because MS provide some easy ways to manage it, it can seem tough though because the documentation covering timeouts specifically is not great, so let me address a few. There are several ways to manage WCF timeouts, the first being in-line with your code.


InventoryService client = new InventoryService();
client.Timeout = 300000;


You see in this code that we identify the "client" as the .NET service reference and then we set the Timeout property to a pretty large number. This is defined in the MSDN as Indicates the time an XML Web service client waits for a synchronous XML Web service request to complete (in milliseconds).
However you will also see in the following configuration section of my services the following:


<binding name="ServiceBinding" receiveTimeout="00:30:00" sendTimeout="00:15:00" closeTimeout="00:15:00" openTimeout="00:05:00">
<reliableSession enabled="true" inactivityTimeout="00:30:00" />
<security mode="None" />
</binding>


So pay close attention:

recieveTimeout is defined as: Gets or sets the amount of time a TcpClient will wait to receive data once a read operation is initiated.

sendTimeout is defined as: Gets or sets the amount of time a TcpClient will wait for a send operation to complete successfully.


open and close only have to do with the opening and closing of the bindings.

If you paid close attention you realized that all three are needed. In the case of the first timeout, the inline code timeout, the is a full timeout that services wait for a request to complete, but the receive timeout only applies once the read operation has started where as the send timeout only applies to the specific send operation. Why do all three apply, amazingly it's only true for the CA feed because of the delay it takes for them to process a sent request in. It would appear as though (not confirmed)that when WCF makes a request it sends back a response saying send complete (makes sense - sendTimeout), then stalls, then begins a receive or read, then a response is sent that a read is completed. So what happens during that stall while the other side is processing. This is what I think is covered by the inline Timeout. Of course yes, the send and receive timeouts apply to the TcpClient while the Timeout inline applies to SoapHttpClientProtocol. These two however tie together to make up a request.