Introduction to the Developer's Intro to Data Science

This collection supports the Channel9/YouTube series called Developer’s Intro to Data Science. This series will target professional developers who are new to Data Science, Machine Learning, and AI.

Useful Resources:
Dev Intro to Data Science 28 Videos:
https://channel9.msdn.com/Series/Dev-Intro-to-Data-Science

Developer’s Intro to Data Science
https://www.aka.ms/DevIntroDS_Learn

HTH. 2020-July-13 by Jacky Chiou

Enables the cipher suites in Cloud Service

Here is the steps:

Step 0: To add the files for a web role:

In Solution Explorer, under Roles in your cloud service project, right-click your web role and select Add > New Folder. Create a folder named bin.
Right-click the bin folder and select Add > Existing Item. Select the files and add it to the bin folder.
To add the files for a worker role:

Right-click your worker role and select Add > Existing Item. Select the files and add it to the role.
When files are added in this way to the role content folder, they’re automatically added to your cloud service package. The files are then deployed to a consistent location on the virtual machine. Repeat this process for each web and worker role in your cloud service so that all roles have a copy of the installer.

Step 1: Create the PowerShell script to enable cipher suite, “TLS_DHE_RSA_WITH_AES_256_GCM_SHA384”.
Use the following code as an example to create a script that enables the cipher suites. For the purposes of this documentation, this script will be named: TLSsettings.ps1. Store this script on your local desktop for easy access in later steps.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
# You can use the -SetCipherOrder (or -sco) option to also set the TLS cipher 
# suite order. Change the cipherorder variable below to the order you want to set on the 
# server. Setting this requires a reboot to take effect.

Param(
[parameter(Mandatory=$false)]
[alias("sco")]
[switch]$SetCipherOrder)

Function DisableRC4 {
   param ( $restart)
  $subkeys = Get-Item -Path "HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL"
  $ciphers = $subkeys.OpenSubKey("Ciphers", $true)

  if($ciphers.SubKeyCount -eq 0) {
    $k1 = $ciphers.CreateSubKey("RC4 128/128")
    $k1.SetValue("Enabled", 0, [Microsoft.Win32.RegistryValueKind]::DWord)
    $restart = $true
    $k2 = $ciphers.CreateSubKey("RC4 64/128")
    $k2.SetValue("Enabled", 0, [Microsoft.Win32.RegistryValueKind]::DWord)
    $k3 = $ciphers.CreateSubKey("RC4 56/128")
    $k3.SetValue("Enabled", 0, [Microsoft.Win32.RegistryValueKind]::DWord)
    $k4 = $ciphers.CreateSubKey("RC4 40/128")
    $k4.SetValue("Enabled", 0, [Microsoft.Win32.RegistryValueKind]::DWord)
  }

  $restart

}

Function Set-CryptoSetting {
  param (
    $keyindex,
    $value,
    $valuedata,
    $valuetype,
    $restart
  )

  # Check for existence of registry key, and create if it does not exist
  If (!(Test-Path -Path $regkeys[$keyindex])) {
    New-Item $regkeys[$keyindex] | Out-Null
  }

  # Get data of registry value, or null if it does not exist
  $val = (Get-ItemProperty -Path $regkeys[$keyindex] -Name $value -ErrorAction SilentlyContinue).$value

  If ($null -eq $val) {
    # Value does not exist - create and set to desired value
    New-ItemProperty -Path $regkeys[$keyindex] -Name $value -Value $valuedata -PropertyType $valuetype | Out-Null
    $restart = $True
    Write-Host "Configuring $regkeys[$keyindex]...."

  } Else {

    # Value does exist - if not equal to desired value, change it
    If ($val -ne $valuedata) {
      Set-ItemProperty -Path $regkeys[$keyindex] -Name $value -Value $valuedata
      $restart = $True
      Write-Host "Configuring $regkeys[$keyindex]..."
    }
  }

  $restart

}

$regkeys = @(
"HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.0",
"HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.0\Client",
"HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.0\Server", #2
"HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.1",
"HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.1\Client", #4
"HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.1\Server",
"HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2",        #6
"HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Client",
"HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Server", #8
"HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL 2.0",
"HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL 2.0\Client", #10
"HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL 2.0\Server",
"HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL 3.0",        #12
"HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL 3.0\Client",
"HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL 3.0\Server", #14
"HKLM:\SOFTWARE\Policies\Microsoft\Cryptography\Configuration\SSL\00010002"
)

Function Set-Windows10PlusCurveOrder {
    param ( $reboot)
    $desiredOrder = "NistP384;NistP256".Split(";")
    If ([Environment]::OSVersion.Version.Major -ge 10) {
        If (!(Test-Path -Path $regkeys[15])) {
            New-Item $regkeys[15] | Out-Null
            $reboot = $True
        }

        $val = (Get-Item -Path $regkeys[15] -ErrorAction SilentlyContinue).GetValue("EccCurves", $null)

        if( $null -eq $val) {
            New-ItemProperty -Path $regkeys[15] -Name EccCurves -Value $desiredOrder -PropertyType MultiString | Out-Null
            $reboot = $True

        } else {

            if ([System.String]::Join(';', $val) -ne [System.String]::Join(';', $desiredOrder)) {
                Write-Host "The original curve order ", `n, $val, `n, "needs to be updated to ", $desiredOrder
                Set-ItemProperty -Path $regkeys[15] -Name EccCurves -Value $desiredOrder
                $reboot = $True
            }
        }
    }

    $reboot

}

If ([Environment]::OSVersion.Version.Major -lt 10) {
  # This is for Windows before 10 
  Write-Host "Configuring Windows before 10..."
  $cipherorder =  "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384,"
  $cipherorder +=  "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P256,"
  $cipherorder += "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384_P384,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256_P256,"
  $cipherorder += "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256,"
  $cipherorder += "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256,"
  $cipherorder += "TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256,"
  $cipherorder += "TLS_RSA_WITH_AES_256_CBC_SHA256,TLS_RSA_WITH_AES_128_CBC_SHA256,"
  $cipherorder += "TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA"

} Else {

# this is for windows 10 or above
Write-Host "Configuring Windows 10+..."
$cipherorder = "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384,"
$cipherorder += "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,"
$cipherorder += "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,"
$cipherorder += "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,"
$cipherorder += "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,"
$cipherorder += "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,"
$cipherorder += "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,"
$cipherorder += "TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256,"
$cipherorder += "TLS_RSA_WITH_AES_256_CBC_SHA256,TLS_RSA_WITH_AES_128_CBC_SHA256,"
$cipherorder += "TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA"
}

# If any settings are changed, this will change to $True and the server will reboot
$reboot = $False

# Check for existence of registry keys (SSL 2.0, SSL 3.0, TLS 1.0, TLS 1.1, TLS 1.2), and create if they do not exist
For ($i = 0; $i -le 14; $i = $i + 1) {
  If (!(Test-Path -Path $regkeys[$i])) {
    New-Item $regkeys[$i] | Out-Null
  }
}

# Ensure SSL 2.0 disabled for client/server
# $reboot = Set-CryptoSetting 10 DisabledByDefault 1 DWord $reboot
# $reboot = Set-CryptoSetting 10 Enabled 0 DWord $reboot
# $reboot = Set-CryptoSetting 11 DisabledByDefault 1 DWord $reboot
# $reboot = Set-CryptoSetting 11 Enabled 0 DWord $reboot

# Ensure SSL 3.0 disabled for client/server
# $reboot = Set-CryptoSetting 13 DisabledByDefault 1 DWord $reboot
# $reboot = Set-CryptoSetting 13 Enabled 0 DWord $reboot
# $reboot = Set-CryptoSetting 14 DisabledByDefault 1 DWord $reboot
# $reboot = Set-CryptoSetting 14 Enabled 0 DWord $reboot

# Ensure TLS 1.0 enabled for client/server
# $reboot = Set-CryptoSetting 1 DisabledByDefault 0 DWord $reboot
# $reboot = Set-CryptoSetting 1 Enabled 1 DWord $reboot
# $reboot = Set-CryptoSetting 2 DisabledByDefault 0 DWord $reboot
# $reboot = Set-CryptoSetting 2 Enabled 1 DWord $reboot

# Ensure TLS 1.1 enabled for client/server
# $reboot = Set-CryptoSetting 4 DisabledByDefault 0 DWord $reboot
# $reboot = Set-CryptoSetting 4 Enabled 1 DWord $reboot
# $reboot = Set-CryptoSetting 5 DisabledByDefault 0 DWord $reboot
# $reboot = Set-CryptoSetting 5 Enabled 1 DWord $reboot

# Ensure TLS 1.2 enabled for client/server
# $reboot = Set-CryptoSetting 7 DisabledByDefault 0 DWord $reboot
# $reboot = Set-CryptoSetting 7 Enabled 1 DWord $reboot
# $reboot = Set-CryptoSetting 8 DisabledByDefault 0 DWord $reboot
# $reboot = Set-CryptoSetting 8 Enabled 1 DWord $reboot
# 
# $reboot = DisableRC4($reboot)

#If ($SetCipherOrder) {
      If (!(Test-Path -Path $regkeys[15])) {
        New-Item $regkeys[15] | Out-Null
        $reboot = $True
      }

      $val = (Get-Item -Path $regkeys[15] -ErrorAction SilentlyContinue).GetValue("Functions", $null)
      Write-Host "val: $val"

      Write-Host "cipherorder: $cipherorder"

      if ($val -ne $cipherorder)
      {
        Write-Host "The original cipher suite order needs to be updated", `n, $val
        Set-ItemProperty -Path $regkeys[15] -Name Functions -Value $cipherorder
        $reboot = $True
      }
# }

$reboot = Set-Windows10PlusCurveOrder $reboot

If ($reboot) {
  # Randomize the reboot timing since it could be run in a large cluster.
  $tick = [System.Int32]([System.DateTime]::Now.Ticks % [System.Int32]::MaxValue)
  $rand = [System.Random]::new($tick)
  
  #$sec = $rand.Next(30, 600)
  $sec = 30 
  Write-Host "Rebooting after", $sec, " second(s)..."
  Write-Host  "shutdown.exe /r /t $sec /c ""Crypto settings changed"" /f /d p:2:4"
  shutdown.exe /r /t $sec /c "Crypto settings changed" /f /d p:2:4

} Else {

  Write-Host "Nothing get updated."
}

Step 2: Create a command file
Create a CMD file named RunTLSSettings.cmd using the below. Store this script on your local desktop for easy access in later steps.

1
2
3
4
echo "Invoking TLSsettings.ps1 on Azure service at %TIME% on %DATE%" >> %TEMP%\StartupLog.txt 2>&1       
PowerShell -ExecutionPolicy Unrestricted .\TLSsettings.ps1 -sco  >> %TEMP%\StartupLog.txt 2>&1

EXIT /B %ERRORLEVEL%

Step 3: Add the startup task to the role’s service definition (csdef)

1
2
3
4
<Startup>
  <Task executionContext="elevated" taskType="simple" commandLine="RunTLSSettings.cmd">
  </Task>
</Startup>

Step 4: Publish & Validate
Now that the above steps have been complete, publish the update to your existing Cloud Service.

For more information:
Troubleshooting applications that don’t support TLS 1.2
https://docs.microsoft.com/en-us/azure/cloud-services/applications-dont-support-tls-1-2

HTH 2020-July-5 by Jacky

How to deploy web app (Python/Django/Linux) via zip

  1. Create resource group (Only run it one time)
    az group create –name “JACKY-RG-TEST” –location “westus”

  2. Create App Service Plan (Only run it one time)
    az appservice plan create –name “jacky-test” –resource-group “JACKY-RG-TEST” –sku S1 –is-linux

  3. Create Web app (Only run it one time)
    az webapp create –resource-group “JACKY-RG-TEST” –plan “jacky-test” –name “jacky-jacky2” –runtime “PYTHON|3.6”

  4. Set application settings SCM_DO_BUILD_DURING_DEPLOYMENT=true
    az webapp config appsettings set -g “JACKY-RG-TEST” -n “jacky-test” –settings SCM_DO_BUILD_DURING_DEPLOYMENT=true

  5. Deploy a ZIP file to the webapp
    az webapp deployment source config-zip –resource-group “JACKY-RG-TEST” –name “jacky-jacky2” –src “C:\temp\application.zip”

  6. Modfiy codes and redeploy a zip to the same webapp
    az webapp deployment source config-zip –resource-group “JACKY-RG-TEST” –name “jacky-jacky2” –src “C:\temp\applicationv2.zip”

  7. Create staging slot
    az webapp deployment slot create –name “jacky-jacky2” –resource-group “JACKY-RG-TEST” –slot “staging” –configuration-source “jacky-jacky2”

  8. Modify codes and deploy it to “staging” slot
    az webapp deployment source config-zip –resource-group “JACKY-RG-TEST” –name “jacky-jacky2” –slot “staging” –src “C:\temp\applicationv3.zip”

  9. After testing. Swap staging to production
    az webapp deployment slot swap -g “JACKY-RG-TEST” -n “jacky-jacky2” –slot staging –target-slot production

You can integrate those commands into your CD tools.

HTH. 2020-June-28 by Jacky

C# console application get secrets from azure key vault service

Step 1: Create a key vault

image

Step 2: Add something to the vault

image

Step 3: Register an App with Azure

  1. Go to Azure Portal, then go to “Azure Active Directory” Section. In the “App registrations” section, click on “New application registration”

    image

  2. Specify the “Name” and “Sign-on URL”(It does not have to be the real one but required.). For “Application Type” must be “Web app/ API” in order to generate the client secret for the app.

    image

  3. Once its finish, you’ll see the “Application ID”. This will be your Client Id.

    image

  4. Next, click on “Settings” button as shown in the figure below. Go to “Keys” section. Then specify the description and choose the expires and your password in “VALUE” . Finally, click on “Save” button

    image

  5. The secret string will be shown once the saving is complete. This will be the “Client Secret” for the App.

    image

  6. Go to your Azure Key Vault. Then, go to “Access Policies” section. Next, Click on “Add New”.

    image

  7. In the “Configure from template” option choose “Key, Secret, & Certificate Management”. Next, “Select Principal” choose the app that was created in the Active Directory.

    image

Step 4: Client Implementation

1
2
3
4
var context = new Microsoft.IdentityModel.Clients.ActiveDirectory.AuthenticationContext(@"https://login.windows.net/Your Tenant");
ClientCredential clientCredential = new ClientCredential("Your Client Id from step 3.3", "Your secret from step 3.5");
var tokenResponse = context.AcquireTokenAsync("https://**vault.azure.net**", clientCredential);
var accessToken = tokenResponse.Result.AccessToken;

HTH. 2020-June-27 by Jacky

Using-Redis-Cache-for-Session-State

It’s often not practical in a real-world cloud app to avoid storing some form of state for a user session, but some approaches impact performance and scalability more than others. If you have to store state, the best solution is to keep the amount of state small and store it in cookies. If that isn’t feasible, the next best solution is to use session state with a provider for distributed, in-memory cache.

Azure Cache for Redis provides a session state provider that you can use to store your session state in-memory with Azure Cache for Redis instead of a SQL Server database.

ASP.NET:
ASP.NET Session State Provider for Azure Cache for Redis
https://docs.microsoft.com/en-us/azure/azure-cache-for-redis/cache-aspnet-session-state-provider

ASP.NET Core:
Session and state management in ASP.NET Core
https://docs.microsoft.com/en-us/aspnet/core/fundamentals/app-state?view=aspnetcore-3.1

2020-June-24 By Jacky

Supported-list-of-negotiable-ciphers-on-our-service-endpoints-in-Microsoft-Azure

This change is to update the SSL cipher suite order and the removal of the RC4 ciphers from the suite.

The Cipher Suite order determines the cipher suites used by the SSL/TLS.

The following cipher suite order is used:

TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384_P384

TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256_P256

TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256

TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256

TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256

TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256

TLS_RSA_WITH_AES_256_GCM_SHA384

TLS_RSA_WITH_AES_128_GCM_SHA256

TLS_RSA_WITH_AES_256_CBC_SHA256

TLS_RSA_WITH_AES_128_CBC_SHA256

TLS_RSA_WITH_AES_256_CBC_SHA

TLS_RSA_WITH_AES_128_CBC_SHA

TLS_RSA_WITH_3DES_EDE_CBC_SHA

For more information:
https://docs.microsoft.com/en-us/archive/blogs/azuresecurity/azure-cipher-suite-change-removes-rc4-support

HTH. 2020-June-3 By Jacky

Hit-ARM-REST-API-limit

Issue:

Hit ARM REST API limit
{“error”:{“code”:”SubscriptionRequestsThrottled”,”message”:”Number of read requests for subscription ‘Your subscription id’ exceeded the limit of ‘12000’ for time interval ‘01:00:00’. Please try again after ‘300’ seconds.”}}

Cause:

These limits are scoped to the security principal (user or application) making the requests and the subscription ID or tenant ID. If your requests come from more than one security principal, your limit across the subscription or tenant is greater than 12,000 and 1,200 per hour.

Suggestions:

1.Increase query interval.

2.Using different security principal (user or application) making the requests. If your application can support it.

i.https://docs.microsoft.com/en-us/azure/active-directory/develop/app-objects-and-service-principals

ii.https://docs.microsoft.com/en-us/powershell/azure/create-azure-service-principal-azureps?view=azps-3.7.0

More Information: https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/request-limits-and-throttling

2020-04-18 HTH. By Jacky.