//Cloud notes from my desk -Maheshk

"Fortunate are those who take the first steps.” ― Paulo Coelho

How to specify VNet details when creating New-AzureBatchPool compute nodes

Recently I had an ask from a developer to check Azure Powershell Command let> New-AzureBatchPool script execution issue. He wanted to specify the VNet details as a parameter to this command so that the batch pool nodes would created with this detail. Unfortunately, we did not have any sample to refer or validate parameter. We spent quite amount of time tweaking the parameter to see the effect but no luck. Lately found a link where these details are explained enough to try on our own.

Please note, failing to follow these condition would throw errors. Suggest to start this https://docs.microsoft.com/en-us/azure/batch/batch-api-basics#pool-network-configuration and also this https://msdn.microsoft.com/library/azure/dn820174.aspx#bk_netconf. Each and every condition in the below list matters.

  • The specified Virtual Network (VNet) must be in the same Azure region as the Azure Batch account.
  • The specified VNet must be in the same subscription as the Azure Batch account.
  • The specified VNet must be a Classic VNet. VNets created via Azure Resource Manager are not supported.
  • The specified subnet should have enough free IP addresses to accommodate the “targetDedicated” property. If the subnet doesn’t have enough free IP addresses, the pool will partially allocate compute nodes, and a resize error will occur.
  • The “MicrosoftAzureBatch” service principal must have the “Classic Virtual Machine Contributor” Role-Based Access Control (RBAC) role for the specified VNet. If the specified RBAC role is not given, the Batch service returns 400 (Bad Request).
  • The specified subnet must allow communication from the Azure Batch service to be able to schedule tasks on the compute nodes. This can be verified by checking if the specified VNet has any associated Network Security Groups (NSG). If communication to the compute nodes in the specified subnet is denied by an NSG, then the Batch service will set the state of the compute nodes to unusable.
  • This property can be specified only for pools created with cloudServiceConfiguration. If this is specified on pools created with the virtualMachineConfiguration property, the Batch service returns 400 (Bad Request).

Working Powershell command let for easy reference:-

Add-AzureRmAccount
Select-AzureRmSubscriptionSubscriptionName “xxxxx Azure xxx xxxx – xxxx”

$batchcontext = Get-AzureRmBatchAccountKeysAccountName nicoloasbatch

$objectvnetconf = New-ObjectTypeName Microsoft.Azure.Commands.Batch.Models.PSNetworkConfiguration

$objectvnetconf.SubnetId = “/subscriptions/xxxxxxxxxxxxxxxxxx/resourceGroups/nicoloasbatch/providers/Microsoft.ClassicNetwork/virtualNetworks/nicolasclassicvnet/subnets/mysubnet1”

$configuration = New-ObjectTypeName “Microsoft.Azure.Commands.Batch.Models.PSCloudServiceConfiguration”ArgumentList @(4,”*”)

New-AzureBatchPool -Id “MikkybatchPool” –VirtualMachineSize “Small” –TargetDedicated 1 –BatchContext $batchcontextNetworkConfiguration $objectvnetconfCloudServiceConfiguration $configuration

How to specify RBAC details, explained in screenshots.

> The “MicrosoftAzureBatch” service principal must have the “Classic Virtual Machine Contributor” Role-Based Access Control (RBAC) role for the specified VNet. If the specified RBAC role is not given, the Batch service returns 400 (Bad Request).

Step1:-

image

Step 2:-

image

Step 3:-

image

How to verify whether it is successfully executed or not.

2

On successful execution…

image

Reference:-

Pool network configuration- https://docs.microsoft.com/en-us/azure/batch/batch-api-basics#pool-network-configuration

Add a pool to an account(networkConfiguration) https://msdn.microsoft.com/library/azure/dn820174.aspx#bk_netconfhttps://msdn.microsoft.com/library/azure/dn820174.aspx#bk_netconf

Thanks to Marie-Magdelaine Nicolas for sharing the powershell command let.

2016-11-25 Posted by | Azure, Azure Batch, PaaS, Powershell | | Leave a comment

Quick tip on Service Fabric Remoting service development

Azure Service Fabric needs no introduction. It is our next gen PaaS offering or also called PaaS v2. It’s been used internally for many years, tested and released as SDK for consumption. Some of the well known offerings like Az Sql, Az DocDB, Skype etc runs on Service Fabric. We already see developer community consuming for their production and hearing lot of goodness.

It is free, any one can download the SDK, develop and run from their laptop or own data center or publish to Azure. It works on windows and Linux as well. It has lot of rich features over the previous PaaS offerings (cloud services) so seeing lot of traction from big companies considering for critical application.

This sample is based on this example:-https://azure.microsoft.com/en-us/documentation/articles/service-fabric-reliable-services-communication-remoting/ 

Service side proj settings: Set the platform target as x64 If you want to use reliable collections, reliable actors APIs, failing to have this set throws as binding exception as below.

System.BadImageFormatException was unhandled
  FileName=Microsoft.ServiceFabric.Services, Version=5.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
  FusionLog=Assembly manager loaded from:  C:WindowsMicrosoft.NETFrameworkv4.0.30319clr.dll
Running under executable  D:Cases_CoderemotingclienttestbinDebugremotingclienttest.vshost.exe
— A detailed error log follows.

 

platform

 

service

 

For client side/calling method, I do not see set up related information in detailed here https://azure.microsoft.com/en-us/documentation/articles/service-fabric-reliable-services-communication-remoting/. I found, these 3 dll’ s has to be referred at client side project for consuming service. I simply copied from service side sample packages folder to calling side proj folder.

image

image

image

client

sample code available – https://1drv.ms/u/s!ApBwDDnGdg5BhNd-KQHtWtaH-sbRcA

2016-11-13 Posted by | .NET, Azure Dev, C#, Microservices, PaaS, ServiceFabric, VS2015 | | Leave a comment

How to list all available VM sizes in a region using .NET (ARM endpoint)

Today, I had a query from a developer asking how to silent authenticate and fetch the list of available VM’s sizes from a particular region using .NET code. They wanted to fetch this detail from their worker role more precisely. They wanted to call the URI as in this article silent authenticated https://msdn.microsoft.com/en-us/library/azure/mt269440.aspx

Method Request URI
GET https://management.azure.com/subscriptions/{subscription-id}/providers/Microsoft.Compute/locations/{location}/vmSizes?api-version={api-version}

On first sight, I thought this as an RDFE endpoint(older portal/SMAPI), but on closer look this turned to be an ARM end point.

How to identify the url is an RDFE/ARM endpoint?

Please note, for RDFE end point we may have to either use certificate based or native client way of authentication.

Since this is an ARM endpoint, we need to follow the service principal way to get the bearer token which is needed for the URI GET call’s.

Step1:-

Perform the following action one by one carefully as in this URL –  https://azure.microsoft.com/en-us/documentation/articles/resource-group-create-service-principal-portal/

  1. Create an Active Directory application
  2. Get client id and authentication key
  3. Get tenant id
  4. Set delegated permissions
  5. Assign application to role

Step2:-

using System;
using System.IO;
using System.Net;
using Microsoft.IdentityModel.Clients.ActiveDirectory;

namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{

            var context = new AuthenticationContext(“https://login.microsoftonline.com/+ “your_tenantid”);
ClientCredential credential = new ClientCredential(“your_client_ID”, “your_client_secret”);
AuthenticationResult result = context.AcquireToken(“
https://management.azure.com/”, credential);
var token = result.CreateAuthorizationHeader().Substring(“Bearer “.Length);

            string uri = @”https://management.azure.com/subscriptions/<your_subscription_Id>/providers/Microsoft.Compute/locations/Southeast Asia/vmSizes?api-version=2015-05-01-preview”;
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(uri);
request.Headers.Add(“Authorization:Bearer ” + token);
var response = request.GetResponse().GetResponseStream();
var output = new StreamReader(response).ReadToEnd();

            Console.WriteLine(output);
}
}
}

P.s:- I have used Adal 2.28.2.795 to avoid async complexities.

on executing,

image

2016-10-19 Posted by | .NET, AAD, ARM, Azure, Azure Dev, C#, PaaS | | 1 Comment

How can I set a message to deadlettered – Service Bus

Today one of my colleague had an interesting problem in setting the SB Queue message as dead letter message. There are ton of sample around how to create DeadLetterQueue(DLQ) or move or read a DLQ message. But did not get a clear picture or documentation around “how one can set the message as Deadlettered through code”. As you know, messages would get automatically moved to DLQ after ‘n’ retry attempt or expiry but in this case he wanted to move it intentionally after some condition in code. For an example, when you see a body contains some text or invalid business code then I wanted to mark it as DeadLetter so that my other piece of code would drink it with recovery logic.

As usual, we started looking at our official documentation, Github pages, SB Explorer, internal discussion alias – there is lot of noise and confusion around in calling Receive/Defer/Deadletter but none gave a closest hint about marking a message as Deadletter. Spent almost couple of hours figuring out this 5 line of code Sad smile more importantly function call “order”. Failing to have this in order would give you hair pulling exception. So focus follow the “order”, say should mark it as Defer() before Receive() and then finally DeadLetter().

Microsoft.ServiceBus.Messaging.MessageNotFoundException was unhandled
  HResult=-2146233088 IsTransient=false
  Message=Failed to lock one or more specified messages. The message does not exist. TrackingId:2d6fb843-0bd8-4b73-9fc0-8f9bffe98ca7_G0_B0, SystemTracker:xxxxxx:QueueXXX, Timestamp:9/2/2016 7:34:42 PM
 

 

//Sample code to receive a message with sequence number and then Deadletter

using Microsoft.ServiceBus.Messaging;

static void Main(string[] args)

        {

var queue = QueueClient.CreateFromConnectionString(“Endpoint=sb://xxx.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=xxxx=”, “queuename”, ReceiveMode.PeekLock);

           BrokeredMessage msg = queue.Receive();

            msg.Defer();

            msg = queue.Receive(msg.SequenceNumber);

            msg.DeadLetter();

        }

Let me know if you think there is a better way than this.

good weekend !

2016-09-03 Posted by | .NET, Azure Dev, PaaS, ServiceBus | | Leave a comment

How to get the connection string for Redis Cache using Azure Redis Mgmt Library

Recently I had this query from an Azure developer asking sample ARM template for creating Redis cache and then get the connection string from the created cache so to pass it to our next resource creation say Azure WebApps . End to end in single ARM template.  I have explored few samples in Github but could not find any end-to end sample to get this done. The one important part is not very clear for me, how to make REST API call within ARM template to get the accesskey for the newly created cache. And the last part about how to play around with strings to create the qualified ConnectionString from the fragmented REST response. I felt quite challenging to get this all done using ARM template without any sample to refer. More importantly I did not find any documentation around ARM capabilities. But I am sure there must be some way to do this by invoking PS or external scripts etc.

Problem statement: 1) Create or update the Redis Cache 2) Get the list of cache and its Access Key 3) Use the details and create the fully qualified ConnectionString

Possible options: Portal/Powershell/CLI/SDK’s. I chose to go .NET way using Azure Redis Management Library 1.2.1(preview)https://www.nuget.org/packages/Microsoft.Azure.Management.Redis/1.2.1-preview.

Here is the code address the problem,


using System;
using Microsoft.Azure;
using StackExchange.Redis;
using Microsoft.Azure.Management.Redis;
using Microsoft.Azure.Management.Redis.Models;
using Microsoft.IdentityModel.Clients.ActiveDirectory;

//Using ADAL library 2.28 to avoid Async complexities
//Using StackExchange Redis dll to resolve "ConfigurationOptions" Dependency


namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            string resourcegroupName = "";
            string cacheName = "";
            string subscriptionId = "";
            string location = "";
            string token = string.Empty;
            string tenantId = "";

            //get the bearer token from AAD by passing your tenantID
            token =GetAuthorizationHeader(tenantId);

            TokenCloudCredentials creds = new TokenCloudCredentials(subscriptionId, token);
            RedisManagementClient redisclient = new RedisManagementClient(creds);

            Console.WriteLine("Do you want to create or update the Redis Cache [y/n]?");
            ConsoleKeyInfo consoleKeyInfo=Console.ReadKey();

            if (consoleKeyInfo.Key.ToString().ToLower() == "y")
            {
                var redisProperties = new RedisProperties();
                redisProperties.Sku = new Sku("Basic", "C", 1);
                var redisParams = new RedisCreateOrUpdateParameters(redisProperties, location);
                //Create or update the existing cache
                redisclient.Redis.CreateOrUpdate(resourcegroupName, cacheName, redisParams);
            }

            //List all the caches under the given resourcegroupname
            RedisListResponse listresponse = redisclient.Redis.List(resourcegroupName);

            string connectionString = string.Empty;
            string existingcachename = string.Empty;
            string existingcachehostname = string.Empty;
            string existingcachepkey = string.Empty;

            foreach (RedisResource redisresource in listresponse.Value)
            {
                Console.WriteLine("DISPLAY EXISTING CACHE..................n");
                existingcachehostname = redisresource.Properties.HostName;
                Console.WriteLine("Redis cachename:"+ existingcachehostname);
                existingcachename = redisresource.Name;
                Console.WriteLine("Redis hostname:" + existingcachename);
                Console.WriteLine("Redis redisversion:" + redisresource.Properties.RedisVersion);
                Console.WriteLine("Redis ProvisioningState" + redisresource.Properties.ProvisioningState);

                //RedisGetResponse response = redisclient.Redis.Get(resourcegroupName, existingcachename);
                Console.WriteLine("PRINTING KEYS..................n");
                RedisListKeysResponse redislistkeysresponse = redisclient.Redis.ListKeys(resourcegroupName, existingcachename);
                Console.WriteLine("PrimaryKey:" + redislistkeysresponse.PrimaryKey);
                Console.WriteLine("SecondaryKey:" + redislistkeysresponse.SecondaryKey);
                Console.WriteLine("RequestId:" + redislistkeysresponse.RequestId);
                Console.WriteLine("StatusCode:" + redislistkeysresponse.StatusCode);

            }
            //for regenerating the Primary and Secondary Key
            // redisclient.Redis.RegenerateKey(resourcegroupName, existingcachename, new RedisRegenerateKeyParameters(RedisKeyType.Primary));

            //to delete the cache. This would take some time to cleanup. 
            //redisclient.Redis.Delete(resourcegroupName, existingcachename);

            //referenced StackExchange.Redis here for only using the ConfigurationOptions
            var configoptions = new ConfigurationOptions();
            configoptions.EndPoints.Add(existingcachehostname);
            configoptions.Password = existingcachepkey;
            configoptions.ConnectTimeout = 10000;
            configoptions.SyncTimeout = 70000;
            configoptions.AbortOnConnectFail = false;
            configoptions.ToString();
            Console.WriteLine("REDIS ConnectionString >"+ configoptions);

            Console.ReadKey();
        }

        private static string GetAuthorizationHeader(string tenantId)
        {
            var authenticationContext = new AuthenticationContext("https://login.microsoftonline.com/"+ tenantId);
            //For clientId, using Powershell wellknown clientId- 1950a258-227b-4e31-a9cf-717495945fc2
            AuthenticationResult resultstr = authenticationContext.AcquireToken("https://management.core.windows.net/",
                            "1950a258-227b-4e31-a9cf-717495945fc2",
                            new Uri("urn:ietf:wg:oauth:2.0:oob"),
                            PromptBehavior.Always);
            Console.WriteLine("Login Successfull..! Access token is ready to use. ");
            return resultstr.AccessToken;
        }
    }
}

Let me know if you find this useful and have details around equivalent ARM for this..

Disclaimer:- The above sample code is provided for the purpose of illustration only and is not intended to be used in a production environment.

2016-07-13 Posted by | .NET, Azure, C#, PaaS, Redis, VS2015 | | 4 Comments

.NET today, tomorrow and future..

This blog post was drafted few weeks ago but stuck in the same state because I was not sure what to add or write on top of what we learned from our recent dotnetConf2016.  Any how today I decided to wrap this so that it will be useful for readers. This post is mainly written for .NET developer who wanted to know where we are heading as  a community.

 

Motivation:- If you notice in recent times, there is a bunch of articles targeting .NET platform and its future. Few trolled about WPF and Winform roadmap. I saw most of them are one sided and simply trolled comparing with recent programming stack like LAMP/MEAN as an alternate. But the true story is, .NET is not going to go away. This applies to Java as well. I have my good old friends still doing WPF consulting work for big banking companies and happy with that. We all need “Growth Mindset” in looking at them.

We may have to agree that our recent startup companies are going 100% open source but when it comes to desktop development still WPF rules.The skill what you acquired from WPF is not going to be waste, because UWP is based on XAML. .NET is growing and evolving faster than ever. If you look at our .NET Foundation, it has got some fantastic open source projects run by community. We are breaking the platform dependencies, IIS, underlying win32 thick references which is the great thing for cross platform movement.  

 

Technology:- Things are changing very rapidly when it comes to programming side especially for the web & mobile technologies. As we march towards mobile first cloud first world, it makes sense to investment towards platform mobility and in cloud. please note, “mobile first” here is nothing to do with our mobile device. It’s about application mobility – should be able to port our app across the device.

Every week/month we see new set of tools/services/JS framework/scripting things comes out constantly and occupies the head line or top slot in tech conferences. One shouldn’t be worrying about new things, we should stay calm and keep learning at least one new languages every year as an investment. If I remember correctly, for last few years we are seeing a trend in IT requirement. Yr 2013 was for Big data, 2014 for Machine learning/deep learning/AI , 2015 for IoT + dockers,  again in 2016 seeing BoT, headless servers, nano servers, cognitive services and what not. Btw, Cloud and OSS is the constant thing for last few years and going to be more talked in coming years as well Smile. It shouldn’t be a matter for us to pick up this in couple of weekends if we know the fundamentals of programming & design standards.

 

 

Roadmap:- If we know the technology roadmap in advance, then it is easy for anyone to refer and suggest for customers. I had this .NET roadmap guide constantly referred, forwarded to architects for newer design consideration but the last one published was in the year 2013 and sadly no update after that.

Get our last guide “.NET Technology Guide for business applications” downloaded from- > here

[P.s: Some of them are already outdated]

 

clip_image002 

 

Few months back, I reached out to Cesar(author of this PDF) seeking clarity on our guidance update. He has promptly shared the preview version but today I see the same guide available for download freely here. What a timing Smile

Microsoft Platform and Tools for Mobile App Development –

https://blogs.msdn.microsoft.com/microsoft_press/2016/06/22/free-ebook-microsoft-platform-and-tools-for-mobile-app-development/

 

9781509304134_thumb

 

Having said all, Thanks to our recent .net Conference for sharing more clarity around .NET future and developments. If you are .NET developer and not seen the updates from our recent dotnetConf 2016 –  then you are in the right place to get the updates through screenshot.

As title says, this is our current landscape. If you look closer, there is “no code sharing” across the platform and also carry the same learning. The new addition to this slide is our “UWP” and “Xamarin” story.

 

image

 

How about .NET tomorrow:- Yes, it allows us to reuse the skills by mastering one/two library not a platform. You can build and reuse the same code across the platform say iOS or Android or to OSX.

Main objective here is to make the “.NET standard library” to grow without updating platforms. It is going to be decoupled so that you can update libraries as you wish.

 

image

 

How is .NET Future looks like: More and more “.NET standard” supported API’s are expected in few months. Complex Libs like App Domains, Full Reflection and Binary Serialization are also getting considered. In near future, one should be able to easily move the code b/w .NET framework or .NET Core or .Xamarin.

 

image

 

.NET future- sub slide (.NET Framework) :-

a) C# – C#7, VB 15 is the talk of the town

b) Going forward- new version of windows 10,

c) Winforms/WPF  –> Store model

d) Winform/WPF convert to Store apps model, use store to push across the organization

 

image

image

image

 

What others are busy learning..

 

clip_image009

 

Closing note: & Key take away:

 

clip_image011

 

image

 

clip_image014

 

Recommended video:-

.NET Conf Day 1 Keynote – Scott Hunter

.NET Conf Day 2 Keynote – Miguel de Icaza

.NET Conf Day 3 Keynote – Scott Hanselman on The State of .NET

 

Watch the event videos here:-https://channel9.msdn.com/Events/dotnetConf/2016

clip_image015

 

Hope this is useful for you..

<update : 6/28/2016>

Another milestone on developer’s world: .NET Core & ASP.NET Core 1.0 got released at Red Hat DevNation. https://channel9.msdn.com/Events/Linux/DevNation-2016

.NET Home Page http://dot.net/

.NET Documentation https://docs.microsoft.com/en-us/dotnet/

Announcement https://blogs.msdn.microsoft.com/dotnet/2016/06/27/announcing-net-core-1-0/

</update>

2016-06-22 Posted by | .NET, ASP.NET 5, Azure Dev, C#, DevOps, OSS, PaaS, Uncategorized, VS2015 | , , , | 2 Comments

%d bloggers like this: