How to write to Windows Azure Website’s application log and more

We struggled with Windows Azure website because we couldn’t write logs. Using Log4Net to write to log file as we used to do, does not work for Azure Website because of the limited file permission. We had to resort to email notification or Virtual Machine when we needed to debug Azure Websites, which was a big headache!

Fortunately, it is all over now. To write to the Application Log of Azure Website, just use System.Diagnostics.Trace name space and use method like TraceInformation, TraceError, TraceWarning to record different levels of log!


Trace.Wrietline("Log Verbose level log");

Trace.TraceInformation ("Log Information level log");

Trace.TraceWarning("Log Warning level log");

Trace.TraceError("Log Error level log");

Then just turn on the Application Logging and select a logging level for that Azure Website.

4-16-2015 5-41-11 PM

With this feature, it becomes much easier to troubleshoot Azure websites. Even better, Microsoft provides this streaming log function from which you can view application logs in REAL-TIME! (New Azure portal only)

4-16-2015 5-29-27 PM

Furthermore, here is something developers will definitely like – this streaming log is also available in Visual Studio, and you can filter the result using Regular Expression! (Latest version Azure SDK is required)

4-16-2015 5-33-57 PM

4-16-2015 5-36-13 PM

Since file logging is supposed to be turned off automatically after 12 hours, if you also want to log into a table storage, not a problem. You can set up a Azure Storage to hold the log.

4-16-2015 5-42-41 PM

Click View settings of the Azure Website in Visual Studio.  In the Log tab, there  will  be a nice table view of the log. I do notice that it uses a lot of memory of the Azure Website. Just something to consider.

4-16-2015 5-43-48 PM

That’s what I know about logging to Azure Websites. Hopefully it is helpful. We use Azure on most of our web applications and I think it is just getting better everyday. Now with the ability to write application log for Azure Website, it just meets all of our needs, but there is still a huge set of exciting features we haven’t used. I look forward to exploring those someday!

Avoid client timeout on WebAPI call

If a WebAPI service takes a long time to complete, the caller, the client side, may timeout waiting for a response. There are several options to solve this issue, you can increase the timeout limit on the client side and web server side, or use a more persistent connection method like WebSocket and SignalR.

Here I propose a different way to avoid the timeout issue that allows the clients side to proactively check the result instead of passively waiting. The idea is pretty intuitive – since the client side times out on waiting for a response, why don’t just disconnect and check back the result later? So the solution is to disconnect with the client side as soon as the request is received and a WebAPI service is provided to enable client to check for the status of the processing. This way the client does not need to keep the connection open until the processing completes or times out. It can simply calls back every 5 seconds to check the result of the processing.

Here I am going to provide some sample code on how to achieve this. First, we create a Task class

public class Task
{
   public string TaskID;
   public Request Request;
   public Status Status;
   public Result Result;

   public void Execute(){
   // Do all sort of calculation and processing here
   // When done, sets the Status of the task to Finished.
   }
}

We then create a class to store tasks and to handle creation, retrieval, and deletion of tasks.

public class TaskManager
{
   private static List<Task> _TaskList = new List<Task>();
   // Create a new instance of Task object
   // and start a new thread to run this task
   public static Result CreateNewTask(Request request){
      var task = New Task(){
                TaskID =  System.Guid.NewGuid().ToString();
                .....
                };
      _TaskList.add(task);
      Thread thread = new Thread(task.Execute);
      task.Status = Status.Processing
      thread.start;
   }
   // This is also useful for purging tasks
   public static void RemoveTask(string taskID)
   public static Task GetTaskByID(string taskID)
}

Up to this point, we have our ‘backend’ built. Now we are going to create a WebAPI method to accept the request from the client side and a method for client side to check a task’s status:

[HttpPost]
public Result ProcessRequest(Request request){
// Submit a request which will be created as a task
// Return the TaskID back to client
}

[HttpGet]
public Result CheckStatus(String taskID){
// Call this every few seconds to check result
}

In this post I shared an idea to avoid client side timeout when calling WebAPI. We chose an approach that enables client side to check the result actively. Although it makes the client side a bit more complex, it is a reliable way to handle WebAPI calls that run a long time and also provides much more flexibility to the client side.

Use Message security on WCF service behind a SSL handler

Configuring WCF web service’s security is just tedious. Microsoft has been trying to make it simple by removing many configuration settings in .NET 4.5 but it could still get messy if you need to touch the security part – There are many bindings and there are Message, Transport, and TransportWithMessageCredentials security modes, each with their own client credential types, not mentions all those authentication modes for Custom Bindings such as AnoymousForCertificate, IssuedTokenForCertificate, IssuedTokenOVerTransport, etc.

Developers are developing WCF web service on the platform from Microsoft. They are the users of the platform in this sense. It’s supposed to be user friendly and intuitive. But I found it is quite difficult to select the right security configuration in different scenarios. Even after you read the documentation from Microsoft carefully, you sill have a very limited idea on how these security modes differentiate from each other.

This post is aimed to cover one small scenario of using WCF security – using a WCF web service with cert-based Message security behind a front-end SSL handler. Often times, your web servers are behind a load balancer that handles all SSL requests and pass in HTTP requests to your IIS. Below is diagram showing the infrastructure.

8-8-2014 4-17-55 PM

At first glance, it seems pretty straightforward – Transport security mode covers SSL security, Message security mode handles message encryption. Hey there is a security mode just for the two modes combined: TransportWithMessageCredentials . We should be able to just use that on both client and service side to achieve what we want, right? However, you just can’t be so optimistic in the world of software development.

This configuration won’t working. First of all, since the service is not really receiving HTTPS requests, Transport mode should not be used. We just need Message security. So below is the correct configuration on service side.

<wsHttpBinding>
     <binding name="WsBindingConfigration">
     <security mode="Message">
      <message clientCredentialType="Certificate" negotiateServiceCredential="true"
algorithmSuite="Default" establishSecurityContext="false"/>
     </security>
    </binding>
  </wsHttpBinding>

The client side is where it gets tricky. I can’t really explain why TransportWihtMessageCredential doesn’t work. Something goes wrong under the hood. But here is the configuration worked for me. Use MutualSSLNegotiated mode and CustomBinding!

<customBinding>
        <binding name="customBinding_CertAuth_ClientService">
           <security authenticationMode="MutualSslNegotiated">
           </security>
           <httpsTransport authenticationScheme="Anonymous"
proxyAuthenticationScheme="Anonymous" useDefaultWebProxy="true">
           </httpsTransport>
        </binding>
</customBinding>

It took me a long while to research and trial and error to finally figure this out. Many development teams don’t have the time to mirror the environments of their clients, which makes finding and troubleshooting issues like this difficult. But hopefully this post can help you out.

 

Deserialization of nested XML in .NET

I used .NET XML serialization a lot in a few projects recently and struggled with marking the nested classes with attributes to serialize correctly. How the serialization works seem to be tricky at the first glance. Although I can get it right with trials and errors. That’s not optimal. It’s time to make a summary of the tips and tricks I have learnt through recent projects.

Let’s jump right into it. I have this class that I want to serialize. It contains a collection of Project objects.


public class MyXML {

 public List<Project> projects { get; set; }

 public class Project
 {
  public int Id { get; set; }
 }
}

I instantiated the class and added 2 projects in it. It’s going to be serialized to below XML. Note that .NET takes the property name projects as the XML array name and uses the object type Project as array item name.

<MyXML>
 <projects>
   <Project>
    <Id>1</Id>
   </Project>
   <Project>
    <Id>2</Id>
   </Project>
 </projects>
</MyXML>

If we want to change the name of array to ‘Ps‘ and array item to ‘P‘, mark the class like this:


public class MyXML {
// If don't have this or left the array name blank, 
// property name "projects" will be used.
 [XmlArray("Ps")]  
// If don't have this or left the array item name blank, 
// class name "Project" will be used.
 [XmlArrayItem("P")]  
public List<Project> projects { get; set; }

 public class Project
 {
  public int Id { get; set; }
 }
}

This generates a XML like below.

<MyXML>
 <Ps>
   <P>
    <Id>1</Id>
   </P>
   <P>
    <Id>2</Id>
   </P>
 </Ps>
</MyXML>

Now this is posing a problem. Since the name of the array item are specified on the collection, what if we want to change the array item’s name based on the member’s type? This is going to be a common request if we are serializing a generic class that can contain any type of collection member. We need a way to specify the array item’s name on the collection’s member instead of on the collection itself.

So let’s take off the [XmlArrayItem(“P”)] first. Now how do we specify the name on the Project class? One may think marking the Project class with [XmlRoot(“P”)] could achieve this, but in fact XmlRoot only works when the object is serialized as the root element. The correct way is to use [XmlType(“P”)] instead. The following class definition allows the XML to be serialized to the same XML above, but we have the flexibility of specifying the array item name for each type of collection member.


public class MyXML {

 [XmlArray("Ps")]
// We will specify this on the definition of the collection's member object
// [XmlArrayItem("P")] 
 public List<Project> projects { get; set; }

// This is NOT going to work
// [XmlRoot("P")]   
// Use XmlType to specify the array item's name
[XmlType("P")]  
public class Project
 {
  public int Id { get; set; }
 }
}

Up to this point, we have been serializing the projects property as a collection with array name at the top level and array item names at the second level. What if we want to place the element of array item under root level? Can we just “ignore” the property projects and directly serialize the collection members? In other words, how do we serialize to below XML?

<MyXML>
 <P>
   <Id>1</Id>
 </P>
 <P>
   <Id>2</Id>
 </P>
</MyXML>

Marking the property [XmlIgnore] seems to be an intuitive way to do it but it will result in the collection projects not being serialized at all. The correct way is to mark the property with [XmlElement()]. The following class definition generates the above XML we want.

public class MyXML {

// This is NOT going to work.
// [XmlIgnore]  
// This makes sure that the collection's members are serialized directly 
// without being the child elements of a array name element.
 [XmlElement("P")]  
 public List<Project> projects { get; set; }

 public class Project
 {
  public int Id { get; set; }
 }
}

Now we are facing a similar challenge here – how to have member specific name if we want to serialize a generic object? The element name it is serialized to is currently specified on the collection projects side, not on the Project object’s  side, which limits us from having an element name for each type of collection member.

In my research I didn’t find a different place to put the attribute or a different attribute to achieve this. However, this can be achieved by XmlAttributeOverrides which can dynamically overwrite attribute on the run!

So what we want to achieve here is just to override the attribute of property projects to be [XmlElement(“A_Dynamic_Name”)]. The following code snippet demonstrates how to use XmlAttributeOverrides to do this.

XmlAttributeOverrides overrides = new XmlAttributeOverrides();
XmlAttributes attrs = new XmlAttributes();

// This is equivalent to the [XmlElement("Anything")]
attrs.XmlElements.Add(new XmlElementAttribute("Anything")); 

// This tells compiler to find MyXML class 
// and find the property 'projects' and override it with the attributes defined above.
overrides.Add(typeof(MyXML), "projects", attrs);

//Add the overrides to the serializer
XmlSerializer xS = new XmlSerializer(typeof(MyXML), overrides); 

This code generates a temporary class definition like below to be serialized.  This allows us to dynamically get the type name of the collection member and use it as the element name.

public class MyXML {

// [XmlElement("Anything")]
public List<Project> projects { get; set; }

 public class Project
 {
   public int Id { get; set; }
 }

}

And this is serialized to this XML:

<MyXML>
 <Anything>
  <Id>1</Id>
 </Anything>
 <Anything>
  <Id>2</Id>
 </Anything>
</MyXML>

As a side note, this can also be used to override the root element with a little modifications.

 XmlAttributes attrs1 = new XmlAttributes();
 attrs1.XmlRoot = new XmlRootAttribute("AnyRootName");
 overrides.Add(typeof(MyXML), attrs1);

Use Reflection in C# to dynamically retrieve properties

One good thing about JavaScript is the simplicity of working on object property. For example,  to traverse all properties of an object:

for(var propNamein myObject){
// Access the property value like this -
// myObject[propName] or myObject.propName
}

You may want to use hasOwnProperty () and isPropertyEnumerable to further filter the properties but it is simple enough.

Also to add or delete a property is just as simple as

Delete myObject.propName; // Delete property
myObject.propName = propValue;  //Add property

The best part is that you can retrieve a property just using a string variable to match its name.

var propName = "color";

var rabbit = {};
rabbit.color = "white";
console.log(rabbit[propName]); // it prints "white"

Of course, a well-developed language such as C# must also have a way to dynamically work on properties – Reflection.

Reflection can achieve many things but here we are only going to talk about property manipulation.

To get a list of properties of an object –

PropertyInfo[] propObjects = myObject.GetType().GetProperties();
// This returns an array of PropertyInfo objects,
// which is basically your property.

To traverse an object’s properties and work with them –

foreach(PropertyInfo prop in propObjects)
{
// access the property's name - prop.Name
// access the property's value - prop.GetValue(myObject)
// to set a property's value - prop.SetValue(myObject, propValue);
}

It is worth mentioning that when retrieving a certain property with name, ignoring case may be needed. To achieve that –

var value =
  myObject
   .GetType()
   .GetProperty(
    propName,
    BindingFlags.IgnoreCase|BindingFlags.Public|BindingFlags.Instance
    )
   .GetValue(myObject);

Note that all three binding flags are necessary, otherwise it won’t work.

Reference:

C# Reflection