eBay Woes

One of the requirements for our eBay implementation was to provide the ability for our marketing department to send and particular set of products up to Channel Advisor. Of course these being non-technical staff, they needed a easy to use and familiar mechanism to create a list of products by their product id, or our internal unique identifier. Excel was the first that came to mind of course and when that recommendation was excepted it reminded me of a class I taught once about loading arrays from Excel. Of course now we have generics so it was an even easier implementation.

Below is the simple code to load a list of product ids from excel into a generic and then use a lambda (LINQ) to get a distinct list of products. The distinct is added because the list is usually copied and pasted. We have what we call skus which are basically the children of products. So if you have a large, medium and small of a shirt, the shirt would be the product id and the skus would be the individual sizes.


StreamReader reader = File.OpenText(@"C:\location of csv.csv");
string[] content = reader.ReadToEnd().Split(new string[1] { Environment.NewLine }, 100000, StringSplitOptions.RemoveEmptyEntries);
reader.Close();
reader.Dispose();
List<Int32> ids = new List<int>();

// ... some special code to handle our product ids

IEnumerable<Row> r = (from p in results.Rows
where ids.Contains(Convert.ToInt32(p["SKU"].Value))
select p).Distinct();


Notice we use IEnumberable as the return set with a Row object. Row is a custom serializeable object used internally for faster communication via our internal SOA architecture and IEnumberable is a faster iteration by just a fraction than iterating over the collection of rows via a foreach loop.

The only slow part of this is the Distinct. This is of course an in memory query which can be intensive and large data sets. It is so far working for about 500K products.

Adventures in WCF and WF

So I set off to rewrite my previously written WSE 3.0 SOA in WCF using WF. The concept based on the original plan was to use WCF obviously to replace the services and use WF rules engine to replace what I called actions. The actions were based on work that Avanade had done here where an action defines the settings that are used for connection information or router forwarding information (an action an object in the framework should take). These definitions were stored in a configuration file.

Well, I got off to a rocky start because I basically took the WCF sample with the router and the Echo Service/Calculator service and changed them to match my needs. First lesson, use the WCF configuration tool. The settings in the configuration files for the services were way off after my rewrite. The configuration tool validated and helped me get them back in sync. The second piece is use the "svcutil" command line tool. Man, this is so undocumented and should be more highlighted somewhere, but was a crucial piece of getting the configs with the right settings.

Couple of comments and I will get into more detail later. Microsoft has finally addressed some of my previous huge concerns for security and compression in the WCF framework. I absolutely think what they did in these two areas in WCF are worlds better than any previous attempts. Good work team!

Next, if you actually read this blog, MS has once again fallen short on documentation for some of the advanced stuff. First, LINQ can not be used in a SOA architecture because some of the core classes in the LINQ framework are not serializable. I know there some folks working on this, so hold your breath. LOL. Also, There is no clear documentation on WF specifically addressing the validation of the rules for framework classes such as the OracleClient. I am still working on figuring this one out so hopefully I can pass along some information for others soon. As I can't find anything via a search engine, I am hoping MS will pull through for me soon, because I can't be the only one on the planet with this issue.