kubernetes删除namespace进入Terminating状态的排查过程

起因：

       安装kubeflow1.0.1版本时，产生一些错误想要删除重新安装，在删除namespace cert-manager时发现命令夯死，而且该命名空间长时间处于Terminating状态。

过程：

     我安照我之前的解决方案解决了此为问题，解决方案链接点此处，在我继续删除istio-system命名空间时又发生类似的问题。

排查：

      查看资源

[[email protected] ~]# kubectl api-resources --namespaced=true -o name | xargs -n 1  kubectl get --show-kind --ignore-not-found -n istio-system 
error: unable to retrieve the complete list of server APIs: webhook.cert-manager.io/v1beta1: the server is currently unable to handle the request
Error from server (MethodNotAllowed): the server does not allow this method on the requested resource
Error from server (MethodNotAllowed): the server does not allow this method on the requested resource

   我们发现webhook.cert-manager.io/v1beta1有错误，我们查看apiservices

[[email protected] ~]# kubectl get apiservices v1beta1.webhook.cert-manager.io -o yaml
apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:
  annotations:
    cert-manager.io/inject-ca-from-secret: cert-manager/cert-manager-webhook-tls
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"apiregistration.k8s.io/v1beta1","kind":"APIService","metadata":{"annotations":{"cert-manager.io/inject-ca-from-secret":"cert-manager/cert-manager-webhook-tls"},"labels":{"app":"webhook","app.kubernetes.io/component":"cert-manager","app.kubernetes.io/instance":"cert-manager-v1.0.0","app.kubernetes.io/managed-by":"kfctl","app.kubernetes.io/name":"cert-manager","app.kubernetes.io/part-of":"kubeflow","app.kubernetes.io/version":"v1.0.0","kustomize.component":"cert-manager"},"name":"v1beta1.webhook.cert-manager.io"},"spec":{"group":"webhook.cert-manager.io","groupPriorityMinimum":1000,"service":{"name":"cert-manager-webhook","namespace":"cert-manager"},"version":"v1beta1","versionPriority":15}}
  creationTimestamp: "2020-04-15T02:06:38Z"
  labels:
    app: webhook
    app.kubernetes.io/component: cert-manager
    app.kubernetes.io/instance: cert-manager-v1.0.0
    app.kubernetes.io/managed-by: kfctl
    app.kubernetes.io/name: cert-manager
    app.kubernetes.io/part-of: kubeflow
    app.kubernetes.io/version: v1.0.0
    kustomize.component: cert-manager
  name: v1beta1.webhook.cert-manager.io
  resourceVersion: "5707612"
  selfLink: /apis/apiregistration.k8s.io/v1/apiservices/v1beta1.webhook.cert-manager.io
  uid: ed5268bd-6121-4b2d-ba93-ac1f1f22a83a
spec:
  caBundle: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURMVENDQWhXZ0F3SUJBZ0lRVUJFQk9jSndNZURUYkVqQm8yWDRHREFOQmdrcWhraUc5dzBCQVFzRkFEQkEKTVJ3d0dnWURWUVFLRXhOalpYSjBMVzFoYm1GblpYSXVjM2x6ZEdWdE1TQXdIZ1lEVlFRREV4ZGpaWEowTFcxaApibUZuWlhJdWQyVmlhRzl2YXk1allUQWVGdzB5TURBME1UVXdNakV5TURkYUZ3MHlOVEEwTVRRd01qRXlNRGRhCk1FQXhIREFhQmdOVkJBb1RFMk5sY25RdGJXRnVZV2RsY2k1emVYTjBaVzB4SURBZUJnTlZCQU1URjJObGNuUXQKYldGdVlXZGxjaTUzWldKb2IyOXJMbU5oTUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQwpBUUVBeGdaWU54WkFtWHZtVis0ZWt3SGVnbEZBVTlULzJwSlhqL2xhMUlqTkprYjA3TW5TZEdPT29HR2xmMnduCk5PRHptelNoZjZYaFB0VlFBdklqcDR1N3VzRkZVeFZma2hhRWQ4TEtHYU4xUTJHWm5BcXVPMVRZUzVZa1lhN24Kc0wyM0FYWFE1YWtiVnVWcnpQUEdyUmF4bHFkbkx6ZitsRk1UNzJDNnE0c0F1a09qR3VFelBsWCs3U1lsaFliSgpzUzIxM3VkWWVKWlJRTjZQZVlnRC9PcnhoaDFGOTBqQVQ4WjIwTnFjMzlmbmtES1RneTg3MmNuaStib3g3SERyCmNQKy9pcGNrZUgzWUcrdGp5SGRZYTAwT3BMc0VNRjJ1U0c3N0NtVGozZEFEWXZoTHRVRGxGS3NPdnc2bVp5ZFYKQng5QmdqcjNGbjFhbm1NeU5qNVN6M082MVFJREFRQUJveU13SVRBT0JnTlZIUThCQWY4RUJBTUNBcVF3RHdZRApWUjBUQVFIL0JBVXdBd0VCL3pBTkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQWtHWDV1QXVYWk0wQTNCM2FreDV4CnNQb2xQVGFuWVA3SHdLa1BPbTJ0alFaUHZNVGdFNjRnMG9jSERaaFFjWE5FcnlZK1ZnSE00dWNQSi9aUVZDZ2wKS1IvcDhYZWVndWFtNDVhb3hDdFFoY0RoaitETFY2MnJPU2ZuaG1jRGxmTm5scXBVbjNoR2tOZnVLd2tSWC94egpSYkRQck11T21kQU1UQkZ1UkUzSkRSbnk4YllkL25jSGxCS1o5WXBVOEwxdXFDSGhhbUMwdjBHZDVTRUJVR09qCmE0N3JnRitpQlFiOW5zN2dPVFJWeHRDZWpSMkdjdEFnNkRaYmswQnI2cmdQTy90RzNsRzNRNlN1VCt2NThHZWYKczJ4K0V0bGJoZDhhM0Z4ajA1WU5rZVhmcm1ObkwzR1AxNjJyV1ljZ3RkZ3dBL1dxL0h5ZU5BVXp1NW8wQ1Q5bQpxQT09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
  group: webhook.cert-manager.io
  groupPriorityMinimum: 1000
  service:
    name: cert-manager-webhook
    namespace: cert-manager
    port: 443
  version: v1beta1
  versionPriority: 15
status:
  conditions:
  - lastTransitionTime: "2020-04-15T02:36:00Z"
    message: service/cert-manager-webhook in "cert-manager" is not present
    reason: ServiceNotFound
    status: "False"
    type: Available

          发现错误service/cert-manager-webhook in "cert-manager" is not present，我们暂时先不管这个错误，我们删除这个报错的apiservice

kubectl delete apiservices v1beta1.webhook.cert-manager.io

        稍等一段时间之后发现istio-system也删除掉了，

 

问题已解决，如有问题请加技术交流群：526855734