Performance optimisation issues for computer vision on heterogeneous and embedded architectures can be categorised into system level and kernel-level optimisations. Existing computer vision libraries such as OpenCV make it difficult to support programmability of user-defined kernels across platforms. In such cases, it is the users’ responsibility to write an optimised kernel for different architectures which is not a trivial task. Providing an abstract model that allows programmers to develop vision applications independently from the underlying architecture can be a viable approach to separating the user-level application definition from the model implementation for different platforms. SYCL is an open standard, C++11-based, cross-platform parallel programming framework, allowing a programmer to execute a SYCL-based applications on different OpenCL-enabled platforms without modifying the code. In this paper, we propose a high-level computer vision framework that generates a compile-time expression tree-based model to address system-level optimisation, which enables running the same applications on different platforms with no modification to the source code. Building our vision framework on top of SYCL, it is possible to support kernel-level optimisations for different OpenCL-capable devices and kernel programmability for user defined kernels. Our performance evaluation of the proposed framework shows that the example pipeline written in our framework is up to 1.35X faster than equivalent pipelines developed using OpenVX and OpenCV.