Sean Dague | bc883df | 2016-08-12 07:21:59 -0400 | [diff] [blame] | 1 | ===================== |
| 2 | DevStack Networking |
| 3 | ===================== |
| 4 | |
| 5 | An important part of the DevStack experience is networking that works |
| 6 | by default for created guests. This might not be optimal for your |
Peter Stachowski | 0dab8d6 | 2016-11-21 20:36:31 +0000 | [diff] [blame] | 7 | particular testing environment, so this document tries its best to |
Sean Dague | bc883df | 2016-08-12 07:21:59 -0400 | [diff] [blame] | 8 | explain what's going on. |
| 9 | |
| 10 | Defaults |
| 11 | ======== |
| 12 | |
| 13 | If you don't specify any configuration you will get the following: |
| 14 | |
| 15 | * neutron (including l3 with openvswitch) |
| 16 | * private project networks for each openstack project |
| 17 | * a floating ip range of 172.24.4.0/24 with the gateway of 172.24.4.1 |
Kevin Benton | 4bfbc29 | 2016-11-15 17:26:05 -0800 | [diff] [blame] | 18 | * the demo project configured with fixed ips on a subnet allocated from |
| 19 | the 10.0.0.0/22 range |
Peter Stachowski | 0dab8d6 | 2016-11-21 20:36:31 +0000 | [diff] [blame] | 20 | * a ``br-ex`` interface controlled by neutron for all its networking |
Sean Dague | bc883df | 2016-08-12 07:21:59 -0400 | [diff] [blame] | 21 | (this is not connected to any physical interfaces). |
Peter Stachowski | 0dab8d6 | 2016-11-21 20:36:31 +0000 | [diff] [blame] | 22 | * DNS resolution for guests based on the resolv.conf for your host |
Sean Dague | bc883df | 2016-08-12 07:21:59 -0400 | [diff] [blame] | 23 | * an ip masq rule that allows created guests to route out |
| 24 | |
| 25 | This creates an environment which is isolated to the single |
| 26 | host. Guests can get to the external network for package |
| 27 | updates. Tempest tests will work in this environment. |
| 28 | |
| 29 | .. note:: |
| 30 | |
| 31 | By default all OpenStack environments have security group rules |
| 32 | which block all inbound packets to guests. If you want to be able |
| 33 | to ssh / ping your created guests you should run the following. |
| 34 | |
| 35 | .. code-block:: bash |
| 36 | |
| 37 | openstack security group rule create --proto icmp --dst-port 0 default |
| 38 | openstack security group rule create --proto tcp --dst-port 22 default |
| 39 | |
| 40 | Locally Accessible Guests |
| 41 | ========================= |
| 42 | |
Jens Harbott | c3b58f5 | 2020-06-08 11:58:29 +0200 | [diff] [blame] | 43 | If you want to make your guests accessible from other machines on your |
Sean Dague | bc883df | 2016-08-12 07:21:59 -0400 | [diff] [blame] | 44 | network, we have to connect ``br-ex`` to a physical interface. |
| 45 | |
| 46 | Dedicated Guest Interface |
| 47 | ------------------------- |
| 48 | |
| 49 | If you have 2 or more interfaces on your devstack server, you can |
| 50 | allocate an interface to neutron to fully manage. This **should not** |
| 51 | be the same interface you use to ssh into the devstack server itself. |
| 52 | |
| 53 | This is done by setting with the ``PUBLIC_INTERFACE`` attribute. |
| 54 | |
| 55 | .. code-block:: bash |
| 56 | |
| 57 | [[local|localrc]] |
| 58 | PUBLIC_INTERFACE=eth1 |
| 59 | |
| 60 | That will put all layer 2 traffic from your guests onto the main |
| 61 | network. When running in this mode the ip masq rule is **not** added |
| 62 | in your devstack, you are responsible for making routing work on your |
| 63 | local network. |
| 64 | |
| 65 | Shared Guest Interface |
| 66 | ---------------------- |
| 67 | |
| 68 | .. warning:: |
| 69 | |
| 70 | This is not a recommended configuration. Because of interactions |
Stephen Finucane | 3362be9 | 2024-08-22 16:24:21 +0100 | [diff] [blame] | 71 | between OVS and bridging, if you reboot your box with active |
John L. Villalovos | 64edfd4 | 2017-08-17 13:21:25 -0700 | [diff] [blame] | 72 | networking you may lose network connectivity to your system. |
Sean Dague | bc883df | 2016-08-12 07:21:59 -0400 | [diff] [blame] | 73 | |
| 74 | If you need your guests accessible on the network, but only have 1 |
| 75 | interface (using something like a NUC), you can share your one |
| 76 | network. But in order for this to work you need to manually set a lot |
| 77 | of addresses, and have them all exactly correct. |
| 78 | |
| 79 | .. code-block:: bash |
| 80 | |
| 81 | [[local|localrc]] |
| 82 | PUBLIC_INTERFACE=eth0 |
| 83 | HOST_IP=10.42.0.52 |
Jens Harbott | c3b58f5 | 2020-06-08 11:58:29 +0200 | [diff] [blame] | 84 | FLOATING_RANGE=10.42.0.0/24 |
Sean Dague | bc883df | 2016-08-12 07:21:59 -0400 | [diff] [blame] | 85 | PUBLIC_NETWORK_GATEWAY=10.42.0.1 |
| 86 | Q_FLOATING_ALLOCATION_POOL=start=10.42.0.250,end=10.42.0.254 |
| 87 | |
| 88 | In order for this scenario to work the floating ip network must match |
| 89 | the default networking on your server. This breaks HOST_IP detection, |
| 90 | as we exclude the floating range by default, so you have to specify |
| 91 | that manually. |
| 92 | |
| 93 | The ``PUBLIC_NETWORK_GATEWAY`` is the gateway that server would normally |
| 94 | use to get off the network. ``Q_FLOATING_ALLOCATION_POOL`` controls |
| 95 | the range of floating ips that will be handed out. As we are sharing |
| 96 | your existing network, you'll want to give it a slice that your local |
| 97 | dhcp server is not allocating. Otherwise you could easily have |
| 98 | conflicting ip addresses, and cause havoc with your local network. |
Kevin Benton | 4bfbc29 | 2016-11-15 17:26:05 -0800 | [diff] [blame] | 99 | |
| 100 | |
| 101 | Private Network Addressing |
| 102 | ========================== |
| 103 | |
| 104 | The private networks addresses are controlled by the ``IPV4_ADDRS_SAFE_TO_USE`` |
| 105 | and the ``IPV6_ADDRS_SAFE_TO_USE`` variables. This allows users to specify one |
| 106 | single variable of safe internal IPs to use that will be referenced whether or |
| 107 | not subnetpools are in use. |
| 108 | |
| 109 | For IPv4, ``FIXED_RANGE`` and ``SUBNETPOOL_PREFIX_V4`` will just default to |
| 110 | the value of ``IPV4_ADDRS_SAFE_TO_USE`` directly. |
| 111 | |
Peter Stachowski | 0dab8d6 | 2016-11-21 20:36:31 +0000 | [diff] [blame] | 112 | For IPv6, ``FIXED_RANGE_V6`` will default to the first /64 of the value of |
Kevin Benton | 4bfbc29 | 2016-11-15 17:26:05 -0800 | [diff] [blame] | 113 | ``IPV6_ADDRS_SAFE_TO_USE``. If ``IPV6_ADDRS_SAFE_TO_USE`` is /64 or smaller, |
Peter Stachowski | 0dab8d6 | 2016-11-21 20:36:31 +0000 | [diff] [blame] | 114 | ``FIXED_RANGE_V6`` will just use the value of that directly. |
Kevin Benton | 4bfbc29 | 2016-11-15 17:26:05 -0800 | [diff] [blame] | 115 | ``SUBNETPOOL_PREFIX_V6`` will just default to the value of |
| 116 | ``IPV6_ADDRS_SAFE_TO_USE`` directly. |
Stephen Finucane | 3362be9 | 2024-08-22 16:24:21 +0100 | [diff] [blame] | 117 | |
| 118 | .. _ssh: |
| 119 | |
| 120 | SSH access to instances |
| 121 | ======================= |
| 122 | |
| 123 | To validate connectivity, you can create an instance using the |
| 124 | ``$PRIVATE_NETWORK_NAME`` network (default: ``private``), create a floating IP |
| 125 | using the ``$PUBLIC_NETWORK_NAME`` network (default: ``public``), and attach |
| 126 | this floating IP to the instance: |
| 127 | |
| 128 | .. code-block:: shell |
| 129 | |
| 130 | openstack keypair create --public-key ~/.ssh/id_rsa.pub test-keypair |
| 131 | openstack server create --network private --key-name test-keypair ... test-server |
| 132 | fip_id=$(openstack floating ip create public -f value -c id) |
| 133 | openstack server add floating ip test-server ${fip_id} |
| 134 | |
| 135 | Once done, ensure you have enabled SSH and ICMP (ping) access for the security |
| 136 | group used for the instance. You can either create a custom security group and |
| 137 | specify it when creating the instance or add it after creation, or you can |
| 138 | modify the ``default`` security group created by default for each project. |
| 139 | Let's do the latter: |
| 140 | |
| 141 | .. code-block:: shell |
| 142 | |
| 143 | openstack security group rule create --proto icmp --dst-port 0 default |
| 144 | openstack security group rule create --proto tcp --dst-port 22 default |
| 145 | |
| 146 | Finally, SSH into the instance. If you used the Cirros instance uploaded by |
| 147 | default, then you can run the following: |
| 148 | |
| 149 | .. code-block:: shell |
| 150 | |
| 151 | openstack server ssh test-server -- -l cirros |
| 152 | |
| 153 | This will connect using the ``cirros`` user and the keypair you configured when |
| 154 | creating the instance. |
Stephen Finucane | 14f60b9 | 2024-08-26 11:56:31 +0100 | [diff] [blame] | 155 | |
| 156 | Remote SSH access to instances |
| 157 | ============================== |
| 158 | |
| 159 | You can also SSH to created instances on your DevStack host from other hosts. |
| 160 | This can be helpful if you are e.g. deploying DevStack in a VM on an existing |
Stephen Finucane | 4993380 | 2024-08-27 12:08:23 +0100 | [diff] [blame] | 161 | cloud and wish to do development on your local machine. There are a few ways to |
| 162 | do this. |
| 163 | |
| 164 | .. rubric:: Configure instances to be locally accessible |
| 165 | |
| 166 | The most obvious way is to configure guests to be locally accessible, as |
| 167 | described `above <Locally Accessible Guests>`__. This has the advantage of |
| 168 | requiring no further effort on the client. However, it is more involved and |
| 169 | requires either support from your cloud or some inadvisable workarounds. |
| 170 | |
| 171 | .. rubric:: Use your DevStack host as a jump host |
| 172 | |
| 173 | You can choose to use your DevStack host as a jump host. To SSH to a instance |
| 174 | this way, pass the standard ``-J`` option to the ``openstack ssh`` / ``ssh`` |
| 175 | command. For example: |
| 176 | |
| 177 | .. code-block:: |
| 178 | |
| 179 | openstack server ssh test-server -- -l cirros -J username@devstack-host |
| 180 | |
| 181 | (where ``test-server`` is name of an existing instance, as described |
| 182 | :ref:`previously <ssh>`, and ``username`` and ``devstack-host`` are the |
| 183 | username and hostname of your DevStack host). |
| 184 | |
| 185 | This can also be configured via your ``~/.ssh/config`` file, making it rather |
| 186 | effortless. However, it only allows SSH access. If you want to access e.g. a |
| 187 | web application on the instance, you will need to configure an SSH tunnel and |
| 188 | forward select ports using the ``-L`` option. For example, to forward HTTP |
| 189 | traffic: |
| 190 | |
| 191 | .. code-block:: |
| 192 | |
| 193 | openstack server ssh test-server -- -l cirros -L 8080:username@devstack-host:80 |
| 194 | |
| 195 | (where ``test-server`` is name of an existing instance, as described |
| 196 | :ref:`previously <ssh>`, and ``username`` and ``devstack-host`` are the |
| 197 | username and hostname of your DevStack host). |
| 198 | |
| 199 | As you can imagine, this can quickly get out of hand, particularly for more |
| 200 | complex guest applications with multiple ports. |
| 201 | |
| 202 | .. rubric:: Use a proxy or VPN tool |
| 203 | |
| 204 | You can use a proxy or VPN tool to enable tunneling for the floating IP |
Stephen Finucane | 14f60b9 | 2024-08-26 11:56:31 +0100 | [diff] [blame] | 205 | address range of the ``$PUBLIC_NETWORK_NAME`` network (default: ``public``) |
Stephen Finucane | 4993380 | 2024-08-27 12:08:23 +0100 | [diff] [blame] | 206 | defined by ``$FLOATING_RANGE`` (default: ``172.24.4.0/24``). There are many |
| 207 | such tools available to do this. For example, we could use a useful utility |
| 208 | called `shuttle`__. To enable tunneling using ``shuttle``, first ensure you |
| 209 | have allowed SSH and HTTP(S) traffic to your DevStack host. Allowing HTTP(S) |
| 210 | traffic is necessary so you can use the OpenStack APIs remotely. How you do |
| 211 | this will depend on where your DevStack host is running. Once this is done, |
| 212 | install ``sshuttle`` on your localhost: |
Stephen Finucane | 14f60b9 | 2024-08-26 11:56:31 +0100 | [diff] [blame] | 213 | |
| 214 | .. code-block:: bash |
| 215 | |
Takashi Kajinami | b609c80 | 2025-01-07 10:51:03 +0900 | [diff] [blame] | 216 | sudo apt-get install sshuttle || dnf install sshuttle |
Stephen Finucane | 14f60b9 | 2024-08-26 11:56:31 +0100 | [diff] [blame] | 217 | |
Stephen Finucane | 4993380 | 2024-08-27 12:08:23 +0100 | [diff] [blame] | 218 | Finally, start ``sshuttle`` on your localhost using the floating IP address |
| 219 | range. For example, assuming you are using the default value for |
| 220 | ``$FLOATING_RANGE``, you can do: |
Stephen Finucane | 14f60b9 | 2024-08-26 11:56:31 +0100 | [diff] [blame] | 221 | |
| 222 | .. code-block:: bash |
| 223 | |
| 224 | sshuttle -r username@devstack-host 172.24.4.0/24 |
| 225 | |
| 226 | (where ``username`` and ``devstack-host`` are the username and hostname of your |
| 227 | DevStack host). |
| 228 | |
Stephen Finucane | 4993380 | 2024-08-27 12:08:23 +0100 | [diff] [blame] | 229 | You should now be able to create an instance and SSH into it: |
| 230 | |
| 231 | .. code-block:: bash |
| 232 | |
| 233 | openstack server ssh test-server -- -l cirros |
| 234 | |
| 235 | (where ``test-server`` is name of an existing instance, as described |
| 236 | :ref:`previously <ssh>`) |
Stephen Finucane | 14f60b9 | 2024-08-26 11:56:31 +0100 | [diff] [blame] | 237 | |
| 238 | .. __: https://github.com/sshuttle/sshuttle |