Saturday, December 25, 2010

Parsing Delicious Export File

With all the brew ha ha going on about how Delicious is going to be dumped I made a backup of my Delicious bookmarks. While I was at it I created a quick utility to parse the export file so I could play with the links and tags.

Key to parsing the file was the HTML Agility Pack. The Bookmark class:

[csharp]
public class Bookmark
{
public string Title { get; set; }
public string Href { get; set; }
public DateTime AddDate { get; set; }
public string AddDateEpoch { get; set; }
public List<string> Tags { get; set; }
public bool IsPrivate { get; set; }
public Bookmark()
{
Tags = new List<string>();
}

public static Bookmark New(HtmlNode node)
{
if (node == null) throw new ArgumentNullException("node");

var bookmark = new Bookmark
{
Title = node.InnerText ?? string.Empty,
Href = node.Attributes["href"].Value ?? string.Empty,
AddDate = FromUnixTime(Convert.ToDouble(node.Attributes["ADD_DATE"].Value ?? "0")),
IsPrivate = (node.Attributes["ADD_DATE"].Value ?? "0").Equals("1")
};

bookmark.Tags.AddRange(GetTags(node.Attributes["tags"].Value ?? string.Empty));

return bookmark;
}

protected static DateTime FromUnixTime(double unixTime)
{
DateTime epoch = new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc);
return epoch.AddSeconds(unixTime);
}

protected static string[] GetTags(string tagList)
{
if(string.IsNullOrEmpty(tagList)) return new string[]{};

return tagList.Trim().Split(',');
}

}
[/csharp]

The tricky part was really just handling the Epoch time for the AddDate field.

Using the class:
[csharp]
var doc = new HtmlDocument();
doc.Load(@"Data\delicious.htm");

var bookmarks = doc.DocumentNode.SelectNodes("//a[@href]").Select(Bookmark.New);

var tags = bookmarks.SelectMany(b => b.Tags).Distinct().OrderBy(t => t);

var output = new StringBuilder();

foreach (var tag in tags)
{
output.AppendLine(tag);
string localTag = tag;
var taggedBookmarks = bookmarks.Where(b => b.Tags.Contains(localTag)).OrderBy(b => b.AddDate);
foreach (var taggedBookmark in taggedBookmarks)
{
output.AppendFormat("\t{0}", taggedBookmark.Title).AppendLine();
}
}

File.WriteAllText("TaggedBookmarks.txt", output.ToString());
Console.WriteLine(output.ToString());

[/csharp]

Friday, November 05, 2010

Moved

This blog has been moved to techshorts.ddpruitt.net

ObservableCollection AddRange

Damon Payne has a post AddRange for ObservableCollection in Silverlight 3. It is pretty short and sweet way to improve performance when batch adding data to an ObservableCollection.

He created a SmartCollection:

[csharp]
public class SmartCollection<T> : ObservableCollection<T>
{
public SmartCollection()
{
_suspendCollectionChangeNotification = false;
}


bool _suspendCollectionChangeNotification;

protected override void OnCollectionChanged(NotifyCollectionChangedEventArgs e)
{
if (!_suspendCollectionChangeNotification)
{
base.OnCollectionChanged(e);
}
}

public void SuspendCollectionChangeNotification()
{
_suspendCollectionChangeNotification = true;
}

public void ResumeCollectionChangeNotification()
{
_suspendCollectionChangeNotification = false;
}


public void AddRange(IEnumerable<T> items)
{
this.SuspendCollectionChangeNotification();
try
{
foreach (var i in items)
base.InsertItem(Count, i);
}
finally
{
this.ResumeCollectionChangeNotification();

var arg = new NotifyCollectionChangedEventArgs(NotifyCollectionChangedAction.Reset);
this.OnCollectionChanged(arg);
}
}
}
[/csharp]

I created a scratch / test app to see if it works:

[xml]
<UserControl
x:Class="SilverlightApplication10.MainPage"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:my="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Data"
mc:Ignorable="d"
d:DesignHeight="300" d:DesignWidth="400">

<Grid x:Name="LayoutRoot" Background="White">
<StackPanel Orientation="Horizontal">
<StackPanel Margin="5">
<StackPanel Orientation="Horizontal" Margin="5">
<TextBlock x:Name="SlowText" Height="50" Width="100"/>
<Button x:Name="SlowButton" Content="Slow" Click="SlowButtonClick"/>
</StackPanel>
<my:DataGrid x:Name="SlowDataView"
Height="300"
AutoGenerateColumns="True"
RowBackground="Cornsilk"
AlternatingRowBackground="LightGray"/>
</StackPanel>

<StackPanel Margin="5">
<StackPanel Orientation="Horizontal" Margin="5">
<TextBlock x:Name="FastText" Height="50" Width="100"/>
<Button x:Name="FastButton" Content="Fast" Click="FastButtonClick"/>
</StackPanel>
<my:DataGrid x:Name="FastDataView"
Height="300"
AutoGenerateColumns="True"
RowBackground="Cornsilk"
AlternatingRowBackground="LightGray"/>
</StackPanel>
</StackPanel>
</Grid>
</UserControl>
[/xml]

[csharp]
public partial class MainPage : UserControl
{
public MainPage()
{
InitializeComponent();
_slowCollection = new ObservableCollection<MyClass>();
SlowDataView.ItemsSource = _slowCollection;

_fastCollection = new SmartCollection<MyClass>();
FastDataView.ItemsSource = _fastCollection;
}

#region Slow

private readonly ObservableCollection<MyClass> _slowCollection;
private void SlowButtonClick(object sender, RoutedEventArgs e)
{
var timeSpan = Time(() => { for (int i = 0; i < 25000; i++) _slowCollection.Add(MyClass.BuildRandom()); });
Dispatcher.BeginInvoke(() => this.SlowText.Text = timeSpan.ToString());
}

#endregion

#region Fast

private readonly SmartCollection<MyClass> _fastCollection;
private void FastButtonClick(object sender, RoutedEventArgs e)
{

var timeSpan = Time(() =>
{
var list = new List<MyClass>();
for (int i = 0; i < 25000; i++) list.Add(MyClass.BuildRandom());
_fastCollection.AddRange(list);
});


Dispatcher.BeginInvoke(() => this.FastText.Text = timeSpan.ToString());
}

#endregion

private static TimeSpan Time(Action a)
{
var start = DateTime.Now;
a();
var end = DateTime.Now;
return end - start;
}
}
[/csharp]

[csharp]
internal class MyClass : INotifyPropertyChanged
{
private static Random _random;

#region Id

private string _id;

public string Id
{
get { return _id; }
set
{
if (value != _id)
{
_id = value;
NotifyPropertyChanged("Id");
}
}
}

#endregion

#region SomeRandomNumber

private int _someRandomNumber;

public int SomeRandomNumber
{
get { return _someRandomNumber; }
set
{
if (value != _someRandomNumber)
{
_someRandomNumber = value;
NotifyPropertyChanged("SomeRandomNumber");
}
}
}

#endregion

#region INotifyPropertyChanged

public event PropertyChangedEventHandler PropertyChanged;

public void NotifyPropertyChanged(string propertyName)
{
var propertyChanged = PropertyChanged;
if (propertyChanged != null)
propertyChanged(this, new PropertyChangedEventArgs(propertyName));
}

#endregion

public static MyClass BuildRandom()
{
if(_random==null)
_random = new Random((int) DateTime.Now.Ticks);

var myclass = new MyClass
{
Id = Guid.NewGuid().ToString("N"),
SomeRandomNumber = _random.Next(1, 100)
};

return myclass;
}
}
[/csharp]

Running the code shows that indeed the AddRange works well to speed up inserts. I was getting about 15 to 25 seconds on the Slow Data and from 3 to 0.09 seconds on the Fast Data. It was interesting to note that if the Slow button was pressed a second time it actually took longer than the first click, but the Fast button had the opposite affect.

Saturday, October 30, 2010

RE:Why is Microsoft suddenly so hot for HTML5? | Betanews

Last week, Gartner predicted that mobile would be a trillion-dollar industry by 2014. It\'s a market Microsoft doesn\'t want to be shut out of.

via Why is Microsoft suddenly so hot for HTML5? | Betanews.

This and my earlier post reinforce my belief that it is time to break the chains of Microsoft.  Meaning it is time for me to stop being a Mort.

Friday, October 29, 2010

RE: Microsoft: Our strategy with Silverlight has shifted | ZDNet

“Silverlight is our development platform for Windows Phone,” he said. Silverlight also has some “sweet spots” in media and line-of-business applications, he said.

But when it comes to touting Silverlight as Microsoft’s vehicle for delivering a cross-platform runtime, “our strategy has shifted,” Muglia told me.

via Microsoft: Our strategy with Silverlight has shifted | ZDNet.

I think it is time to move away from the Microsoft platform.  I'm getting tired of having to learn a new framework every year just to keep up with their changes.

I've been using Silverlight at work and I like it but if there is not going to be any support or they totally re-target the platform for just phones, whats the point?

Time for python - html5 - JavaScript stack.

Asynchronous Programming in C#

Making Asynchronous Programming Easy

Asynchronous Programming in C#

[csharp]public async void AsyncIntroParallel()
{
Task<string> page1 = new WebClient().DownloadStringTaskAsync(new Uri("http://www.weather.gov"));
Task<string> page2 = new WebClient().DownloadStringTaskAsync(new Uri("http://www.weather.gov/climate/"));
Task<string> page3 = new WebClient().DownloadStringTaskAsync(new Uri("http://www.weather.gov/rss/"));

WriteLinePageTitle(await page1);
WriteLinePageTitle(await page2);
WriteLinePageTitle(await page3);
}[/csharp]

Async CTP has been released. Note: This is not automatically Multi-Threading! From the CTP:

Merely sticking the "Async" modifier will not make a method run on the
background thread. That is correct and by design. If you want to run code on a
background thread, use TaskEx.Run(). Please read the overview for an explanation
of asynchrony.

When Does Hosting Your Website in the Cloud Make Sense?

When Does Hosting Your Website in the Cloud Make Sense?.

Discussion of when to use cloud vs own metal.  Short answer:

  • When you need that extra boost in bandwidth due to being slammed with requests.


Basically don't just jump to the cloud because it is 'cool', use the cloud if it saves you money.

Wednesday, October 27, 2010

UOW: FileHelpers

The utility of the week is FileHelpers, a .Net library that works with delimited and fixed length files. It helps to import and export data from files, strings or streams.

I found this recently as I needed to process several CSV files but I did not want to write by hand a parser. Been there, done that way too many times so I did a search and what do you know- someone smarter than me already did it.

What I like about it is the wizard that comes with the library that helps to build plain old c# classes to work with the data rows. In about twenty minutes I had a working CSV processing engine complete with intellisense.

The library isBSD License, GNU Library or Lesser General Public License (LGPL)

Threading AutoResetEvent Example

[csharp]public class Program
{
private static System.Threading.AutoResetEvent stopFlag = new System.Threading.AutoResetEvent(false);
public static void Main()
{
ServiceHost svh = new ServiceHost(typeof(ServiceImplementation));
svh.AddServiceEndpoint(
typeof(WCFSimple.Contract.IService),
new NetTcpBinding(),
"net.tcp://localhost:8000");
svh.Open();
Console.WriteLine("SERVER - Running...");
stopFlag.WaitOne();
Console.WriteLine("SERVER - Shutting down...");
svh.Close();
Console.WriteLine("SERVER - Shut down!");
}
public static void Stop()
{
stopFlag.Set();
}
}[/csharp]

Monday, September 27, 2010

Coding Horror: Because Everyone Needs a Router


Coding Horror: Because Everyone Needs a Router: "Buffalo Nfiniti Wireless-N High Power Router ($80)"

Jeff Atwood discusses routers and what he did to build a new one. He used the Buffalo Nfiniti Wireless-N and DD-WRT to replace his aging yet functional DGL-4500.

Friday, September 24, 2010

The SMAQ stack for big data - O'Reilly Radar

The SMAQ stack for big data - O'Reilly Radar: "'Big data' is data that becomes large enough that it cannot be processed using conventional methods. Creators of web search engines were among the first to confront this problem. Today, social networks, mobile phones, sensors and science contribute to petabytes of data created daily."

What other types of large data sets can we use this approach on? The Oil & Gas industry deals with tons of data so maybe SMAQ can be used with it.

Sunday, September 05, 2010

Use Linq and Reflection to Activate Interface and Execute a Method.

I created a simple ICommand interface similar to the one in System.Windows.Input:
[csharp]interface ICommand
{
string Description { get; }
bool CanExecute(object parameter);
void Execute();
}[/csharp]
In my project I created a bunch of classes that implement my ICommand interface and I wanted to find and execute them all.  I came up with two ways to do this, using a List<ICommand> and reflection.

List<ICommand>


This is very direct and easy but as I added new commands the list began to grow. Also I had to remember to add my commands to the list as I created them.
[csharp]
string separator = new string('=', 100);
var commandsToExec = new List<ICommand>
{
new Commands.TestEntityConnectionStringBuilderCommand(),
new Commands.BenchmarkingExampleCommand(),
new Commands.FastTokenizerBenchmarkCommand(),
new Commands.LinqToXmlTest01()
};
foreach (var cmd in commandsToExec)
if (cmd.CanExecute(null))
{
Console.WriteLine("{0}{1}{2}{1}{0}{1}", separator, Environment.NewLine, cmd.Description);
cmd.Execute();
Console.WriteLine("{1}{0}{1}{1}", separator, Environment.NewLine);
}
[/csharp]

Reflection


Using linq I find all the types in my Assembly that implements the ICommand interface.  I then loop through the results, create an instance of the object and cast it as a ICommand and then call the execute method.
[csharp]
var commandClasses = from type in Assembly.GetExecutingAssembly().GetTypes()
where type.GetInterfaces().Contains(typeof(ICommand))
select type;
foreach (var commnadType in commandClasses)
{
var cmd = Activator.CreateInstance(commnadType) as ICommand;
if (cmd.CanExecute(null))
{
Console.WriteLine("{0}{1}{2}{1}{0}{1}", separator, Environment.NewLine, cmd.Description);
cmd.Execute();
Console.WriteLine("{1}{0}{1}{1}", separator, Environment.NewLine);
}
}
[/csharp]

Tuesday, August 24, 2010

InfoQ: WAF and Caliburn: 2 WPF Application Frameworks

InfoQ: WAF and Caliburn: 2 WPF Application Frameworks

Two MVVM frameworks.

WAF is backed by the patterns & practices team (waf.codeplex.com).

Caliburn though is said to be more mature (caliburnproject.org).

A third framework mentioned in the notes is mvvmlight (mvvmlight.codeplex.com).