Tuesday, January 9, 2018

WCF - Concurrency, InstanceContextMode and SessionMode & request handling behavior

Though WebAPI is penetrating the service world in .Net WCF still has a space in .Net. The main reason is the support for protocols other than http and there are lot of legacy systems out there which has to be maintained till the world fully moved to Cloud native.

WCF is easy to develop but when it comes to production, there are so many internal things to be known than just writing the service. The major area to be known before putting any WCF service is how the service is going to serve requests. Will there be one service object? How many requests will be processed in parallel? In there are more than one requests getting processed how many service objects will be there to process? Is it one to one or one to many? If it is one service object processing  many requests what is means to thread safety? Service object here means the object of class which implements the service contract interface? Hope everyone knows who creates the object of implementation class.

There are so many articles out there which explains how the service process requests. So this post is to consolidate all in one place to avoid googling.

MSDN Article 1 & Article 2 - These are the best article if someone have enough understanding about the concept. If someone is really fresh to WCF this is not a great place to start

Sivaprasad Koirala on Instancing & concurrency - This is good place to start for freshers. He explains with diagrams and there is a good sample which can be used to understand the behavior. But it talks only instancing and concurrency. When instancing combined with Concurrency mode and Session, the behavior will change.

If we are in hurry and have to understand the behavior quickly the best site is given below.

https://whinery.wordpress.com/2014/07/29/wcf-sessions-instancing-and-concurrency-explained/

It explains with combinations and tell what happens in each combination. But the unlucky blog post has no comments till now even after 3 years.

It is not ending here. There is throttling behavior which might need to be tweaked. Also the security mode which gets enabled automatically for some bindings though we don't need it and reduce throughput.

Tuesday, January 2, 2018

ReST based Web Sites & ReSTful Navigation

Introduction

ReST is highly considered as architectural pattern for integrating systems. It mainly uses APIs
point to resource and do operation on those. The URL often follows a cleaner hierarchical path pattern without much key value pairs than conventional key value pair based URL schemes. People often follow the ReST based URL format for APIs but not widely accepted for web sites.

This post aims to investigate on bringing this ReST based URL schemes to web sites similar to APIs.

Why should I do create web sites as ReSTful URLs

The same benefit of ReST based API URLs applies here as well. The URL will be easy to remember. It is easy to have separation of concerns. New features can be totally implemented separately. using their own area / virtual directories. No need to mix with existing screens even it they are related. Simple and short URLs than lengthy story telling URLs.

eg: www.mycompany.com/employeelist.aspx?empId=1 can be easily represented via
www.mycompany.com/employees/1

Why there are not much web sites following ReSTful pattern

One reason could be the difficulty to follow the pattern. If it is product company the development team will get more freedom to select URLs. Again if the product owners don't know what is ReST and the advantages they may influence the URL pattern. In the other side the consulting industry is heavily driven by client demands. Though regular clients may not ask for particular URLs, semi technical clients may ask.

Some tips for ReSTful web site URLS

Below are some tips to design web site URLs in resource oriented way

No operation oriented screens

The screens should be pointing to resources to display those. For example the below URL displays the employee resource.

www.mycompany.com/employees/1

Lists and details screens

If the URLs display the resource how to edit them? Which screen to edit those resources? The better way is to use lists and details mechanism.

If we display the resources in a list those can be edited in the same list itself and save using a button. Here the single page application (SPA) concept helps us than navigating to an edit page.

Circular navigation

If the resources have circular relation the navigation may also becomes circular.
For example, the employee page may show the department where he belongs to as hyper link. Clicing that will navigate to dept page where it lists the employees in that dept including the manager. Going to the manager page may display employees under him and clicking on the same employee's link there will end up in same employees page where we started.

Multiple navigation paths

Similarly there would be multiple navigation paths to reach one resource. For example the home page of company may show the departments and various project it is doing currently. Navigating to department as well as project may end up in same employee page.

A powerful search experience to navigate

If a resource is buried under the hierarchy, it would be difficult to find it without multiple clicks. So better to have a search mechanism where the resource can be searched and navigated on clicking the associated URLs.

http://mikeschinkel.com/blog/welldesignedurlsarebeautiful/

Tuesday, December 19, 2017

Azure @ Enterprise - Is Azure Functions Enterprise ready?

"It Doesn't Work That Way in Enterprise". This is a famous quote when we work in enterprise. Many things which does in a tick at startups never work that way in enterprise. Coming to Azure development, if we want to create a new Azure Function it might be just an entry in ARM template in startup and next deployment will have the Azure Function, but it might be 2 weeks process in Enterprise. Sometimes the Azure Function might have to be created manually based on tickets in Enterprise after multi level approvals and refer its name in ARM.

The focus of this post is 'Is Azure Function Enterprise ready?' Azure Functions is the flagship product from Microsoft to meet the demands of Serverless world.

Signing the assemblies

The basic thing in .Net is to sign our assemblies before shipping them. If we use third party libraries and those contain unsigned assemblies, we cannot sign our assemblies. 

Why we are discussing this basics with Functions? The current Azure Functions SDK which is coming via nuget package has references / dependencies which are not signed!!!

At the time of writing this post, Visual Studio 2017 (15.4.0) creates Azure Function projects which has references to unsigned dlls.

To be more precise, SDK has dependency to Microsoft.Azure.WebJobs.Extensions.Http which has the assembly with same name. If we open the assembly in JustDecompile, we can see the below.

Microsoft.Azure.WebJobs.Extensions.Http, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null

It is reader choice to decide whether a SDK with unsigned assembly is Enterprise ready.

Beta dependencies of Azure Functions SDK

No need to explain in detail. The Function SDK has dependency Microsoft.Azure.WebJobs.Extensions.Http which is still in beta1 stage. Yes its the same which has unsigned assembly referred above. The latest Functions.SDK version available at the time of writing this post in nuget repo is 1.0.7 and it still has this beta reference. Refer the below link for dependencies. The SDK is in production but not the dependencies.


Another thing Enterprise loves to put into production.

Pricing & vNet capability

If a service is internal ie called by other front end services or scheduled tasks etc..., enterprise never want to put it publicly. The solution in Azure Function to have internal service is to use the AppServiceEnvironment+AppServicePlan combination. What we loose there is the cost factor. ASE has fixed money in it and never pay per use.

Execution timeout of Functions

At the of writing this post the max timeout of a Function is 10 mins if it is in consumption plan ie the real pay per use model. But Enterprise may need to execute long running operations greater than 10 mins if there is queuing system or scheduled processes and those should be strictly internal. In order to achieve it they can never use pay per use model of Functions, instead use either ASE+Function or WebJobs. It would be considered as road block by enterprise users.

It seems Microsoft wanted to have an answer to Amazon Lambda in the Serverless world and someone hacked Azure WebJobs to become Azure Functions. But its really excellent idea for start ups to reduce time to market and reduce upfront investments.

Comparing Amazon v/s Azure Function for enterprise use is altogether separate topic.

Disclaimer

The Azure evolves at a fast pace and the facts might not be relevant in future.

Tuesday, December 12, 2017

Excel Function to get stock price without Yahoo finance API

Yahoo finance API was free and helping so many people to get their things done. As per the internet forums many were doing business around that API. One of the best use case is to pull stock price to Excel spreadsheets.

Unfortunately the API is discontinued now. So what is the alternative? Its nothing but another free API. A promising one is Alphavantage. We have to get their API key before using the API. Its just free. No idea whether they throttle or make the service priced later. The web site is given below.

https://www.alphavantage.co/

Below goes the Excel VBA code for having an Excel Function which accept the stock symbol and returns the price.

Public Function StockQuote(strTicker As String) As String
    Dim key As String
    key = "<YOUR KEY FROM Alphavantage>"
    If IsMissing(strTicker) Then
        StockQuote = "No input"
        Exit Function
    End If
    Dim strURL As String, strCSV As String, strRows() As String, strColumns() As String
    Dim dbClose As Double

    'Compile the request URL with needed data.
    strURL = "https://www.alphavantage.co/query?function=TIME_SERIES_DAILY" & _
    "&symbol=" & strTicker & _
    "&interval=15min&outputsize=compact&datatype=csv&" & _
    "apikey=" & key
    
    On Error GoTo catch
        Set http = CreateObject("MSXML2.XMLHTTP")
        http.Open "GET", strURL, False
        http.Send
        strCSV = http.responseText
    
        ' The most recent information is in row 2, just below the table headings.
        ' The price close is the 5th entry
        strRows() = Split(strCSV, Chr(10)) ' split the CSV into rows
        strColumns = Split(strRows(1), ",") ' split the relevant row into columns. 1 means 2nd row, starting at index 0
        dbClose = strColumns(4) ' 4 means: 5th position, starting at index 0
        StockQuote = dbClose
        Set http = Nothing
        Exit Function
catch:
        MsgBox (Err.Description)
End Function

Thanks to the original coder who posted the snippet for Yahoo API. 

Tuesday, December 5, 2017

Waiting on multiple C# .Net awaits

Introduction

Async and Await makes developers life easy without the callback hell in asynchronous programming. But it is equally harmful, if it is in the hands of typewriting coders. Mainly those who don't know how things are working can use async and await in wrong way. This post examines one of such scenario and how to avoid it.

Lets consider the below example. There are 2 independent web service calls to be made and once result is available, do some operation using the results from both the async calls.

private static async Task<string> GetFirstTask(HttpClient client)
{
            Log(nameof(GetFirstTask));
            return await client.GetStringAsync("http://httpbin.org/drip?numbytes=3&duration=3&code=200");
}
private static async Task<string> GetSecondTask(HttpClient client)
{
            Log(nameof(GetSecondTask));
            return await client.GetStringAsync("http://httpbin.org/drip?numbytes=6&duration=6&code=200");
}
private void Process(string first, string second)
{
            Log($"{nameof(Process)} - Length of first is {first.Length} & second is {second.Length}");
}
private static void Log(string msg)
{
            Console.WriteLine($"Thread {Thread.CurrentThread.ManagedThreadId}, Time {DateTime.UtcNow.ToLongTimeString()}, Message {msg}");
}

The first 2 methods returns 2 generic Task<string>. The URL is using httpbin.org which is a hosted service for testing purpose. The duration in the query string controls the delay. Meaning the response will be coming after that duration. Just to avoid Thread.Sleep(). The Process() just display it's parameters.

The normal way

Below is the code we can see more from new async await users.

async internal Task TestNormal_TheBadMethod()
{
    HttpClient client = new HttpClient();
    string firstrequest = await GetFirstTask(client);
    string secondrequest = await GetSecondTask(client);

    Process(firstrequest, secondrequest);
}

The output might be something like below.

Thread 1, Time 8:47:00 PM, Message GetFirstTask
Thread 9, Time 8:47:02 PM, Message GetSecondTask
Thread 7, Time 8:47:07 PM, Message Process - Length of first is 3 & second is 6

Problem

The line where GetFirstTask() is called will wait till the result is obtained. ie wait for 3 seconds to get response from web service. The second task will start only the first is completed. Clearly sequential.

await at method invocation

This is another way developers try.

async internal Task TestViaAwaitAtFunctionCall_StillBad()
{
    Log(nameof(TestViaAwaitAtFunctionCall_StillBad));
    HttpClient client = new HttpClient();
    Process(await GetFirstTask(client), await GetSecondTask(client));
}

Output will look as follows.

Thread 1, Time 8:49:22 PM, Message GetFirstTask
Thread 7, Time 8:49:25 PM, Message GetSecondTask
Thread 9, Time 8:49:30 PM, Message Process - Length of first is 3 & second is 6

Problem

In other languages await keyword at function invocation might make it parallel. But in C# its still sequential. It wait for first await and then process second.

Making it run parallel

So what is the solution? Both the Tasks should be created before we wait for their results. So those tasks will run in parallel. Once await is called, they just give the result if available or wait till the result is available. So the total time is the highest time, not sum of all wait times. Below code snippets does it.

private async Task TestViaTasks_Good()
{
            Log(nameof(TestViaTasks_Good));
            HttpClient client = new HttpClient();
            Task<string> firstrequest = GetFirstTask(client);
            Task<string> secondrequest = GetSecondTask(client);
            Process(await firstrequest, await secondrequest);
}

Output looks below.

Thread 1, Time 8:55:43 PM, Message GetFirstTask
Thread 1, Time 8:55:43 PM, Message GetSecondTask
Thread 8, Time 8:55:48 PM, Message Process - Length of first is 3 & second is 6

Here the Tasks are created before any waits are places on them. So they worked in parallel.

Will this work when second call dependent on first call's result

Not at all. Because the second call cannot start without the result from first call. So this has to be sequential.

More reading

https://stackoverflow.com/questions/33825436/when-do-multiple-awaits-make-sense
https://stackoverflow.com/questions/36976810/effects-of-using-multiple-awaits-in-the-same-method

Tuesday, November 21, 2017

Azure @ Enterprise - Architecture to secure PaaS

When enterprise wants to use somebody else's computer, yes cloud/Azure is essentially just somebody else's computer, they want to make sure it is secure same to their on-premise environment. At least enterprise wants to delay the attack. Never think any security solution is final. Security measures are just about 'when' the attack will happen.

Multi-tenancy and Enterprise

Cloud is all about multi-tenancy. We have to share with others and enterprise always have doubt on it. When we share, will the other guy stole our assets or attack me? Whatever the cloud vendors assure that the cloud is secure and nothing gets leaked between the tenants, enterprise don't want to be just another tenant. They want their own space.

One way is to setup private cloud infrastructure inside enterprise. But it has upfront investment and lose the glory of auto scaling. So that is not the best options unless the security is the first concern.

Enter virtual networks

In order to give own space to enterprise or anybody who don't want to be another tenant, cloud providers started giving virtual networks. It ensures that our cloud resources are accessible only within the boundary of vNet and to have external communication separate gateways/tunnels are needed. They also support seamless connection to the enterprise's on premise network via VPN. That way enterprise can just lift and shift their classic virtual machine based solutions to Cloud and slowly adopt the cloud native features

In Amazon, it is called as VPC (Virtual Private Cloud) and in Azure it is simply virtual network or vNet for short.

vNet limits in Azure

Though there is vNet in Azure not all platform services are vNet ready. For example, we can put a virtual machine easily into vNet but Storage Account cannot. (Feature to limit Storage Account access to vNet is beta at the time of writing). Platform service such as AppService can be hosted inside vNet using a costly feature called AppServiceEnvironment.

Authenticating to platform services

Enterprise normally don't want to keep any secrets in application configuration. eg: Password to database connection should never be kept in web.config in case of an ASP.Net web application. Instead the app has to run under a specific IIS application pool and app pool user will be added to the database server hence app can use windows authentication. Or the authenticated user can be delegated to database directly. Again depends on the nature of application. We can ask then, what is the login password of app pool user? Is one or persons knowing it not a problem in enterprise?

In enterprise people changes often. If the identity is with people, it is easier to manage than keeping password in config. There are chances that someone in support may send the config file in mail or keep in \\shared drive etc... If it is app pool user and the password is with single team who installs the application, it is relatively safe. At least that is what enterprise believe.

Coming back to Azure. Not all Azure resources/platform services(PaaS) support windows/active directory based authentication. The application must preset password when it authenticate against the PaaS. So it demands apps to keep secrets such as passwords and connection strings with password in it. 

Enter KeyVault

Microsoft suggests to use a PaaS called KeyVault to keep secrets. It has HSM (Hardware Security Modules) and much more features. Then the question from enterprise turns out to be "Does KeyVault support vNet?" Answer is No. 
Should we keep the KeyVault's connection string with password in application's config? Then what is the difference between keeping database's connection string with password v/s KeyVault connction string with password in application's config?

So how do we protect KeyVault. If enterprise needs to be just another tenant in KeyVault, how to ensure the security? This post is about one way how can we protect connection to KeyVault which will protect other resources.

Architecture

The key message is simple here. 
In order to connect to KeyVault use certificate instead of password in application's config
Rest is circus we do to get all working together. We can use Azure Function to talk to KeyVault and vNet to protect that function so that nobody from outside can access that Function. Only that Function needs to know the certificate and registered as Azure AD App etc...

Diagram

This is the diagram which shows how the things are put together.

How it works

KeyVault

Since all the Azure resources are not compatible with vNet and Active Directory based authentication, we cannot avoid keep the passwords and secrets. It is advised to keep the secrets in the KeyVault than in app or web.config.

This way the secrets will always be with one team who installs the application. 

Azure AD App

In order to access KeyVault we need to have an Azure Active Directory Application. KeyVault accepts the AD token to get authenticated. 

KeyVault Function/Web App

This is the only component which knows how to connect to Azure KeyVault via Azure AD. It is called by other components in the system whenever they need any secret. This must be the only one registered with AzureAD App.

Alternative is to register all the components as Azure AD App and have the certificate with them. So they can access KeyVault directly

ASE

ASE is required to make sure that the KeyVault Azure Function is accessible within the boundary of enterprise's private area ie vNet. This makes the KeyVault Function secure without accepting security credential in every http request.

Tuesday, November 14, 2017

C# async and await with Thread static

This is continuation of below post about async and await. That post discuss about how the async and await evolved and the basics. The same sample context is used in this post as well. So better read below post before continuing.

http://joymonscode.blogspot.com/2015/06/c-async-and-await-programming-model.html

Async Await and calling thread behavior

There is no objection that it simplified the reasoning about the code. But it may cause trouble, if we implement without understanding how it works. Let us see one example below.

public void Main()
{
           Console.WriteLine($"Main() - Thread Id - {Thread.CurrentThread.ManagedThreadId}");
           for (int counter = 1; counter < 5; counter++)
           {
               if (counter % 3 == 0)
               {
                   WriteFactorialAsyncUsingAwait(counter)
               }
               else
               {
                   Console.WriteLine(counter);
               }
           }
}
private async Task WriteFactorialAsyncUsingAwait(int facno)
{
    Console.WriteLine($"WriteFactorialAsyncUsingAwait() - Thread Id - {Thread.CurrentThread.ManagedThreadId} - Begin");
    int result = await Task.Run(()=> FindFactorialWithSimulatedDelay(facno));
    Console.WriteLine($"WriteFactorialAsyncUsingAwait() - Thread Id - {Thread.CurrentThread.ManagedThreadId} - Factorial of {facno} is {result}");
}


Guess what would be the thread ids printed from WriteFactorialAsyncUsingAwait(). Will those be same?

Those who says same, be prepared to spend nights and weekend debugging. Especially if you have something ThreadStatic before and after await. Below goes the output.

Main() - Thread Id - 1
1
2
WriteFactorialAsyncUsingAwait() - Thread Id - 1 - Begin
4
WriteFactorialAsyncUsingAwait() - Thread Id - 3 - Factorial of 3 is 6

In the code the await is executed in separate thread similar to its Task<> equivalent

Thread.ContinueWith and calling thread behavior

 Lets see what is its Task<> based implementation.

public void Main()
{
    Console.WriteLine($"Main() - Thread Id - {Thread.CurrentThread.ManagedThreadId}");
    for (int counter = 1; counter < 5; counter++)
    {
        if (counter % 3 == 0)
        {
            WriteFactorialAsyncUsingTask(counter);
        }
        else
        {
            Console.WriteLine(counter);
        }
    }
    Console.ReadLine();
}
private void WriteFactorialAsyncUsingTask(int no)
{
    Console.WriteLine($"WriteFactorialAsyncUsingTask() - Thread Id - {Thread.CurrentThread.ManagedThreadId} - Begin");
    Task<int> task=Task.Run<int>(() =>
    {
        int result = FindFactorialWithSimulatedDelay(no);
        return result;
    });
    task.ContinueWith(new Action<Task<int>>((input) =>
    {
        Console.WriteLine($"WriteFactorialAsyncUsingTask() - Thread Id - {Thread.CurrentThread.ManagedThreadId} - Factorial of {no} is {input.Result}");
    }));
}

See the output it is working the same way as of async await model.

Main() - Thread Id - 1
1
2
WriteFactorialAsyncUsingTask() - Thread Id - 1 - Begin
4
WriteFactorialAsyncUsingTask() - Thread Id - 4 - Factorial of 3 is 6

Why the consuming code is written inside ContinueWith({}) callback instead of reading the result from Task.Result property? If it is not via ContinueWith({}), the execution wait on task.Result line, hence the outer loop cannot move to next item. We loose all the benefits of Task then. Below goes the code for task.Result access and see how it blocks the outer loop from executing in parallel.



public void Main()
{
    Console.WriteLine($"Main() - Thread Id - {Thread.CurrentThread.ManagedThreadId}");
    for (int counter = 1; counter < 5; counter++)
    {
        if (counter % 3 == 0)
        {
            WriteFactorialAsyncUsingTask(counter);
        }
        else
        {
            Console.WriteLine(counter);
        }
    }
    Console.ReadLine();
}
private void WriteFactorialAsyncUsingTask(int no)
{
    Console.WriteLine($"WriteFactorialAsyncUsingTask() - Thread Id - {Thread.CurrentThread.ManagedThreadId} - Begin");
    Task<int> task=Task.Run<int>(() =>
    {
        int result = FindFactorialWithSimulatedDelay(no);
        return result;
    });
    Console.WriteLine($"WriteFactorialAsyncUsingTask() - Thread Id - {Thread.CurrentThread.ManagedThreadId} - End - Task Result - {task.Result}");
}

The output below shows clearly that the 4 is processed from the loop only after the 3 is processed. We lost the parallelism. The thread ids are same before and after.

Main() - Thread Id - 1
1
2
WriteFactorialAsyncUsingTask() - Thread Id - 1 - Begin
WriteFactorialAsyncUsingTask() - Thread Id - 1 - End - Task Result - 6
4

Moral of the story

Though async await seems easy to use, usage without understanding will take away our sleep and weekends.