why ctypes.addressof() give diifferent result using python2 and python3

Question

Recently I'm trying to transfer from python2 to python3,in my codes there is some job about read data form hardware that have a .py interface file call foreign .dll lib. The data is shared by memory between .dll and python routine,specifically speaking, ctypes.creat_string_buffer() and ctypes.addressof(), which run correctly under python2.7 env,but give unexpected result under python3.6, the reason seems to be that ctypes.addressof() give huge difference address value, I wonder what's the reason?

'''python2.7 output of addressof()

(base) C:\Users\Administrator>python
Python 2.7.15 |Anaconda, Inc.| (default, May  1 2018, 18:37:09) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> s = 128
>>> p = create_string_buffer(s)
>>> print(addressof(p))
50341488
>>> hex(addressof(p))
'0x3002670L'

'''

'''python3.6 output of addressof()

(base) C:\Users\Administrator>conda activate py36

(py36) C:\Users\Administrator>python
Python 3.6.8 |Anaconda, Inc.| (default, Feb 21 2019, 18:30:04) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> s = 128
>>> p = create_string_buffer(s)
>>> print(addressof(p))
>>> 2241150277680
>>> hex(addressof(p))
>>> '0x209cef75830'

'''

In my opinion,the output of addressof() funtion under python2 and python3 should be approximate,but in fact it's not。Some one who can help me to point what's wrong with the routine,or with me , appreciatively!

Answer 1

[Python 3.Docs]: ctypes - A foreign function library for Python.

ctypes.addressof doesn't have anything to do with the values, it simply reports the address of the underlying C buffer for the ctypes object.

It's about:

• ctypes.create_string_buffer (which returns a ctypes.c_char array)
• How ctypes allocates objects: PyCData_MallocBuffer which in turn calls PyMem_Malloc (check [Python 3.Docs]: Memory Management for a bit more details)
• PyMem_Malloc (in Release mode):
  • For Python 2, it's a malloc wrapper
  • For Python 3, things are more complex, allocations occurs in blocks, user contexts are present, ..., but in the end it still relies on malloc (or one of its family functions)

Output:

[cfati@CFATI-5510-0:C:\WINDOWS\system32]> sopr.bat
*** Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ***

[prompt]>
[prompt]> set _PYTHON2="e:\Work\Dev\VEnvs\py_064_02.07.15_test0\Scripts\python.exe"

[prompt]> set _PYTHON3="e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe"

[prompt]> rem PYTHON 3 - CTYPES

[prompt]> %_PYTHON3% -c "from ctypes import *; print(hex(addressof(create_string_buffer(128))))"
0x25652521230

[prompt]> %_PYTHON3% -c "from ctypes import *; print(hex(addressof(create_string_buffer(128))))"
0x25c61371230

[prompt]> %_PYTHON3% -c "from ctypes import *; print(hex(addressof(create_string_buffer(128))))"
0x20a3dfd1230

[prompt]> rem PYTHON 2 - CTYPES

[prompt]> %_PYTHON2% -c "from ctypes import *; print(hex(addressof(create_string_buffer(128))))"
0x2f5f3b0L

[prompt]> %_PYTHON2% -c "from ctypes import *; print(hex(addressof(create_string_buffer(128))))"
0x262f3b0L

[prompt]> %_PYTHON2% -c "from ctypes import *; print(hex(addressof(create_string_buffer(128))))"
0x313f3b0L

[prompt]>
[prompt]> rem PYTHON 3 - REGULAR

[prompt]> %_PYTHON3% -c "print(hex(id(1234567)))"
0x1df0d6af5b0

[prompt]> %_PYTHON3% -c "print(hex(id(1234567)))"
0x14f39e7f5b0

[prompt]> %_PYTHON3% -c "print(hex(id(1234567)))"
0x1f61453f5b0

[prompt]> rem PYTHON 2 - REGULAR

[prompt]> %_PYTHON2% -c "print(hex(id(1234567)))"
0x2a2ca90L

[prompt]> %_PYTHON2% -c "print(hex(id(1234567)))"
0x325c6a0L

[prompt]> %_PYTHON2% -c "print(hex(id(1234567)))"
0x28bbec0L

As seen, the memory addresses pattern is not related to ctypes objects, but to all (and the address is different every time).

Note that relying on the fact that an address pointing to some memory that has been allocated by malloc (or friends) is equal (or close) to some value, is a bad idea. Code relying on such assumptions, is bad (might say that's the equivalent of Undefined Behavior).