Skip to main content

FileSystemWatcher With the BlockingCollection

While working with the FileSystemWatcher I found that if too many files were created the built in buffer will overflowed and files will be skipped.  After much research I found out about the Producer-Consumer Problem.  Then I found that .Net 4 has the BlockingCollection which helps solve the issue.  But how to use it with the FileSystemWatcher?

On StackOverflow I found Making PLINQ and BlockingCollection work together.  I'm not so interested in the PLINQ issue but this is a great example of using The BlockingCollection with FileSystemWatcher.

[csharp]
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Threading;

namespace ConsoleApplication4
{
    public class Program
    {
        private const string Folder = "C:\\Temp\\InputData";

        static void Main(string[] args) {

            var cts = new CancellationTokenSource();
            foreach (var obj in Input(cts.Token))
                Console.WriteLine(obj);
        }

        public static IEnumerable<object> Input(CancellationToken cancellationToken) {
            var fileList = new BlockingCollection<string>();

            var watcher = new FileSystemWatcher(Folder);
            watcher.Created += (source, e) => {
                if (cancellationToken.IsCancellationRequested)
                    watcher.EnableRaisingEvents = false;
                else if (Path.GetFileName(e.FullPath) == "STOP") {
                    watcher.EnableRaisingEvents = false;
                    fileList.CompleteAdding();
                    File.Delete(e.FullPath);
                } else
                    fileList.Add(e.FullPath);
            };
            watcher.EnableRaisingEvents = true;

            return from file in
                       fileList
                            .GetConsumingEnumerable(cancellationToken)
                            .AsParallel()
                            .WithMergeOptions(ParallelMergeOptions.NotBuffered)
                            .WithCancellation(cancellationToken)
                            .WithDegreeOfParallelism(5)
                   let obj = CreateMyObject(file)
                   select obj;
        }

        private static object CreateMyObject(string file) {
            return file;
        }
    }
}
[/csharp]

Comments

Popular posts from this blog

Json for jqGrid from ASP.Net MVC

jqGrid takes a specific format for its json (taken from jqGrid documentation): [js]{ total: "xxx",page: "yyy", records: "zzz", rows : [ {id:"1", cell:["cell11", "cell12", "cell13"]}, {id:"2", cell:["cell21", "cell22", "cell23"]}, ... ]}[/js] The tags mean the following: total - Total number of Pages. page - Current page Index. records - Total number of records in the rows group. rows - An array with the data plus an identifier. id - The unique row identifier, needs to be an int from what I have found. cell - An array of the data for the grid. The ASP.Net MVC framework has the JsonResult response type which we can use to populate the jqGrid. As an example I created a Person model and a method to return some data: [csharp] public class Person { public int ID { get; set; } public string Name { get; set; } public DateTime Birthday { get; set; } } public I

Changing Typed DataSet Connection String

I was working on a WinForm app that connected to a MS Access database. Yeah, Access sucks but I didn't have a choice in the matter. The app itself is used to import a bunch of CSV files into the Access database. It is more of a utility program and it has going through several variations, from being a simple hand driven command line tool to being GUI driven. The command line version was all hand controlled. I had to go in and update configuration files to point to the CSV files and the MDB database. That got old fast so I decided to make a GUI version that would allow me to pick the MDB file and each of the CSV files to import. Picking and using the CSV files was easy, it was changing the connection string for the MDB that proved to be the hardest. I am using strongly typed datasets in VS2005. If you have ever worked with them you find out soon that the connection string gets saved with the project in the app.config file, even if it is a seperate DAL dll project. My guess is

Remote Controlled RoboTank

This is my version of the ever popular to build RoboTank. It uses an Arduino Mega 2560 with the AdaFruit motor shield and an XBee S1 to communicate to the DFRobot Gamepad. The sketch for the RoboTank makes use of the AFMotor.h to drive the motors and includes a serial parser to read and process the commands coming from the Gamepad. Robotank-Sketch.zip DFRobot Wireless Joystick