Overwriting A Datafeed. Can It Be Done??

[coda]

My dropshipper offers an inventory datafeed that I can d/l to excel every day. However, that feed is full of issues such as: Typos, small images, and obsolete retail prices which they aren’t planning on fixing any time soon. No, they aren’t fly by night. Too successful, too busy to be bothered with small stuff like that, I suppose.

I thought of offering less items and attempting to do the whole thing manually, by re-offering the same products each day and then I could have bigger images (the company has bigger images I can use, but for some reason simply does not have them available on their datafeed. Go figure). I can also clean up the typos in the descriptions, and collect true retail prices so customers would get a better sense of the discount they were getting from shopping with me.

The problem is I am bucking the business model and it has to be problematic. The biggest issue: I’ll be severely limited to the number of products I can manually check against the datafeed each day to be sure I am not out of stock on a product. How many products can I safely do this for each and every day? How can my business ever grow? I chose a business that has thousands of products and I am limiting myself to maybe a few hundred? My competitors will all have thousands. How can I compete? I don't see myself checking to see if thousands of items are in stock each day. That's not what I envision as a business owner. It would kill all the pleasure of the concept for me and I really don't want an employee and what a tedious job to stick somebody with!

My question is this: Is there any way I can make these changes and save them and still use the datafeed in an automated fashion. Is there some type of command I can use (I know nothing of programming!) that can overwrite the data in the datafeed that will allow for the larger image or the saved retail price etc.? Naturally, I would hire someone to do this for me, that is, program it one time.

Is there another solution, or am I doomed to be either a small insignificant site or another cookie cutter site with small images, old retail prices, and too many typos, that looks just like every other site that sells the same products?

Help me please!!!

[mobilebadboy]

You're manually making these changes and then manually putting the products on your site? If so then you are doing alot of tedious work.

While not in the dropshipping business, I deal with pretty much the same thing with the merchants I work with in affiliate marketing. I get their feeds in, most common, a .csv file. I have a PHP script that will pull the feed from wherever (their ftp server, etc) and then it will automatically read through the file and import everything into a mySQL database. Of course, like you, the feeds are often far from perfect.

In the same file (I have one for each merchant) I will add further changes I want to make (replace, remove, add, etc) once its in the database. Of course there will always be new changes that I have to watch out for, but once I make a change then the script will always at least make those changes every time I run it. Some feeds only require a few changes, some require alot of babysitting.

Then I just code my site to query the database and pull whatever products I've told it to pull.

In order to do this though, you will need to know a little about php/mySQL (or other coding/database language). Or as you said, hire someone to do it. Any kind of automation, or semi-automation, that helps alleviate some work always helps.

[coda]

MobileBadBoy....I don't do any programming, so if I am understanding you correctly, I can basically turn the feed into a database and then the database is programmed to use a certain description, or image for a certain product instead of the one in the datafeed? That is only done one time for each product? What happens when a particular product is out of stock one day? On that day, it won't even be present in the datafeed at all. That's the beauty of the automation. If the product is out of stock, it won't populate my website. Would it do so using the formula your suggesting?

[Blue Cat Buxton]

You have hit on the real problem with datafeeds - they are often incomplete, poorly written, stock items change etc. You can get round some of the problems with automation if you are prepared to put in some time, and you will end up with a more unique offering. For example you could have a table with your product descriptions, images, reviews etc, with a field for a product id reference that is common to the merchants datafeed.

Then you only use the changing in stock / current price elements from the data feed to update your website. You will still find that product numbers change though, and there is a lot of work in getting your original data correct (but possibly worth it as you will have a content rich resource).

In the automation process you could have the script tell you when something isn't quite right - say a product in the merchants feed that does not tie up with your database.

I had something similar running in the past, but the really frustrating part is how often merchants change the format of their feed, and then your automation script just stops working correctly and you have to start all over.

[mobilebadboy]

I guess it depends who you work with. I have dozens of these scripts running for dozens of merchants and rarely make many changes to them once I get them like I like them. Of course through the affiliate networks all of the feeds are conformed to one standard setup. The only problem arises when a merchant decides to move info from one column to another (which I rarely see happen and that I don't see happen with the "smart" ones). A quick change in either my script or my site code fixes that.

I couldn't imagine managing them all manually.

coda,

My scripts run once a day and always retrieve a fresh feed. If it doesn't exist in the feed it doesn't exist in the database. The script takes the same info, line by line as you see it in excel, and stores it line by line in the database. I can query a single product or multiple products based on category, however I want to do it.

Since the process can take seconds to minutes, depending on the size of the feed, I always have it create a temporary table first, import everything to that and make changes, then drop the "in use" table and rename the temp table to the in use table name. Mainly to minimize site disruption.

[Chris]

You're running under the assumption that you have to use the entire feed, you do not.

You can easily set things up so that your site keeps images, descriptions, etc that you have edited the same and uses the datafeed only to update stock levels and prices.

This does require programming depending on your platform, so I guess since you don't program you'd need to hire someone, but it is a very easy thing to do.

[coda]

If I don't have to use the entire datafeed that is wonderful news! Thank you and everyone that took the time to respond. Now can you possibly point me to the best place I can find a programmer to discuss this with?

[Chris]

scriptlance.com maybe.