Skip to main content

Corporate Add-on

note

Previously, corporate environments was flagged as C3* (Confidential Level 3). If you are interested to understand why the naming has changed, have a look at the Corporate Addon Article on MCS blog"

note

03/03/2023 - Conditional access policy is not yet deployed on production. You will receive a notification by email in the coming weeks to notify you when the policy will be deployed.

Access​

How to access your corporate addon cluster

Context​

Trustnest is a cloud platform deliberately exposed on internet to facilitate:

  • the accesses from the users to Thales New Generation of Products (SaaS)
  • the ingestion of data coming from Thales Products (ex: IoT)

In certain cases (look at the use cases section), this wide exposition is not desired. To do so, a corporate addon can be enabled during the deployment to restrict the endpoint allowed.

In the following diagrams, we have separated 2 type of access:

  • the developer who need to access to the AKS Master API, Log analytics and Ingresses
  • the end user who only need to access to Ingresses

img

Now in terms of access:

if you come from a workstation connected with RIE network, you will have these flows allowed / blocked:

img

And if you work from a TNAP:

img

note

Detailed architecture views about corporate addon with k8saas are available with authentication.

Use cases​

  • All the users are Thales employees coming from the corporate Network
  • The confidentiality of the data is too high to accept the usual risk of data leakage
  • Development teams are more comfortable developing while disconnected from the internet

What to do ?​

By default, K8SAAS is deployed without the corporate addon to benefit from a full internet experience. To enable it, you need to specifically ask for it when you subscribe to a k8saas offer. From March 2023, this addon is available in Innovate and Industrialise service offers.

How to enable the corporate addon during the creation ?

In postIT, you have to tick the "Corporate Addon" option, like in the following screenshot:

img

warning

The deployment of the corporate addon is possible only at the initialisation of the cluster because it requires specific IP ranges.

Known limitations​

azure auth plugin with AKS​

You could see this warning message if you haven't use the kubelogin command:

az aks get-credentials --resource-group k8saas-***** --name k8saas-*****
A different object named clusterUser_k8saas-*****_k8saas-***** already exists in your kubeconfig file.
Overwrite? (y/n): y
Merged "k8saas-*****" as current context in /Users/loicjardin/.kube/config

kubectl get namespace -A
W0303 20:31:27.221525   44256 azure.go:92] WARNING: the azure auth plugin is deprecated in v1.22+, unavailable in v1.26+; use https://github.com/Azure/kubelogin instead.
To learn more, consult https://kubernetes.io/docs/reference/access-authn-authz/authentication/#client-go-credential-plugins
To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code DY5N9NYT2 to authenticate.

If you continue, you should see this page:

img

And then the following error:

img

Conclusion:

  • This is issue is known. To get around you should use kubelogin command as described in the following lines:
az aks get-credentials --resource-group k8saas-***** --name k8saas-*****
A different object named clusterUser_k8saas-*****_k8saas-***** already exists in your kubeconfig file.
Overwrite? (y/n): y
Merged "k8saas-*****" as current context in /Users/loicjardin/.kube/config

kubelogin convert-kubeconfig -l azurecli

kubectl get namespace -A
W0303 20:31:27.221525   44256 azure.go:92] WARNING: the azure auth plugin is deprecated in v1.22+, unavailable in v1.26+; use https://github.com/Azure/kubelogin instead.
To learn more, consult https://kubernetes.io/docs/reference/access-authn-authz/authentication/#client-go-credential-plugins
To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code DY5N9NYT2 to authenticate.

Azure Cloud Shell​

Even if you are using a laptop connected to RIE network, Cloud Shell session runs in a container in a Microsoft network.

source: https://learn.microsoft.com/en-us/azure/cloud-shell/private-vnet

img

Additional tests have been performed with Business Lines developers, Trustnest teams (including experts in network, identity and kubernetes). We can confirm that cloud shell is not using RIE networks as outbound flow.

Conclusion:

  • cloud shell is not located into the RIE network
  • cloud shell is not mastered by Thales IS/IT
  • So cloud shell is not allowed to access to corporate environments

Self Sign Certificates from AKS API Server​

AKS is semi-managed services. A part of the system is managed by Azure like the API server.

img

source: https://learn.microsoft.com/en-us/azure/aks/concepts-clusters-workloads

AKS API server uses a self-signed certificate. This is a limitation from AKS Microsoft.

For instance if you check the certificate from your browser, you should see this:

img

Microsoft provides a way to use a Custom certificate authority in AKS but in any case: the domain used by AKS API server is not configurable. So even if you provide a valid certificate, your browser will reject it because the domain used by AKS (managed by MS) can't match your domain.

Note: Once your k8saas instance is deployed, the url will not change.

Conclusion:

  • Thales Digital Factory has no choice to accept this limitation.
  • From RIE workstations, you probably need to ask security teams to allow your specific URL from the Thales Proxy.

Pomerium / SSO new generation is not supported​

Pomerium is based on an ingress controller that manages ingress configurations. By default, pomerium is configured to work with a public ingress controller. However, k8saas with corporate addon does not have any public ingress controller and enable only Private Ingress Exposition.

Conclusion:

  • From Jan 2023, k8saas has an opened issue and engineering teams are working on pomerium to support private ingress.
  • If you are looking for solution to protect your application, we recommend using the old way: Built-in SSO to allow all Thales employees (Deprecated) and configure your own Azure Enterprise Application.

HOWTO​

HOWTO be c3* approved ?​

k8saas corporate addon is an enabler to accelerate the process to be approved c3. Only an Information System Security Committee (ISSC) can certify if a perimeter is approved to manipulate c3 data. During this committee, the owner of the service must show:

  • the purpose of the solution (users/data/ what is does)
  • the c3 requirements implementation (best practices)
  • the threat Model with a Risk Analysis Assessment

In addition to this information, a Technical Architecture is often appreciated, and c3 requirements proofs are mandatory.

To help passing the ISSC, k8saas provides:

  • a Technical Architecture and Security Document - TASD
  • a list of c3 requirements already covers or what to do (or not do)

HOWTO Use Trustnest C3 Artificatory from k8saas instance with a corporate addon​

note

Standard k8saas instance can't access to Trustnest C3 Artifactory. Even if you try to use the right certificate, the network access is not allowed.

For now, c3 artifactory use self signed certificate and prevents containerd to pull images.

Label is usefull with job/pod workload that are not retryable. By placing a node affinity on pod/job it's avoid pulling error when new node is adding (migration/autoscale). The job/pod stay in pending phase and wait to fill condition to be schedule on ready node.

on pod spec you shoul add:

affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: c3-root-certificate
            operator: In
            values:
            - ready

on gitlab runner

  config.template.toml:   |
    [[runners]]
      [runners.kubernetes]
        [runners.kubernetes.affinity]
          [runners.kubernetes.affinity.node_affinity]
            [runners.kubernetes.affinity.node_affinity.required_during_scheduling_ignored_during_execution]
              [[runners.kubernetes.affinity.node_affinity.required_during_scheduling_ignored_during_execution.node_selector_terms]]
                [[runners.kubernetes.affinity.node_affinity.required_during_scheduling_ignored_during_execution.node_selector_terms.match_expressions]]
                  key = "c3-root-certificate"
                  operator = "In"
                  values = ["ready"]

HOWTO disable Conditional Access Policy ?​

By default, conditional access policy must be enforced. For an operational constraint, and with a security approval, the k8saas operation team can disable this feature.

To do so, please:

  • raise a ticket to postIT (generic demand)
  • precise the purpose of your request: Get a temporary security exemption on corporate addon with k8saas
  • give your security partner & CISO contacts
  • justify the exemption providing factual blockers (screenshots and network flows diagrams)

In case of doubt, a security partner will contact you to deep dive on your request.

Troubleshooting​

Your device is required to be managed to access this resource.​

When you are requesting to AKS API, you can have the following issue:

Error: failed to get token: expected an empty error but received: AzureCLICredential: ERROR: AADSTS530003: Your device is required to be managed to access this resource.

If you have the previous issue, it means that your TNAP is not compliant. Please perform the following check:

  • Zscaler should run (bleu icon on your top bar)
  • Zscaler module should be properly authenticated (like in the following 2 screenshots)

img

img

If Zscaler is OK, we recommend to re-connect your MDM, just click on "connect" like on the following screenshot

img

Next steps​